Jul 17 13:43:14.333992 ip-10-0-137-163 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 17 13:43:14.334007 ip-10-0-137-163 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 17 13:43:14.334016 ip-10-0-137-163 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 17 13:43:14.334321 ip-10-0-137-163 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 17 13:43:23.862459 ip-10-0-137-163 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Jul 17 13:43:23.862473 ip-10-0-137-163 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3f1dd12bc5404d59b21b5e353e338a8c --
Jul 17 13:45:14.186260 ip-10-0-137-163 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 13:45:14.538809 ip-10-0-137-163 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 13:45:14.538809 ip-10-0-137-163 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 13:45:14.538809 ip-10-0-137-163 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 13:45:14.538809 ip-10-0-137-163 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 13:45:14.538809 ip-10-0-137-163 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 13:45:14.540950 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.540830    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 13:45:14.545557 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545543    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 13:45:14.545557 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545557    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545561    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545564    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545567    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545570    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545573    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545576    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545579    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545581    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545584    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545587    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545589    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545592    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545596    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545599    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545602    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545605    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545607    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545610    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 13:45:14.545617 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545613    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545615    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545618    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545621    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545624    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545627    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545629    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545632    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545635    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545637    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545640    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545642    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545645    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545647    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545650    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545653    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545656    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545658    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545660    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545663    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 13:45:14.546119 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545665    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545668    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545670    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545673    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545675    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545677    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545680    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545682    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545685    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545687    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545689    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545692    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545694    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545697    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545701    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545703    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545706    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545708    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545710    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 13:45:14.546588 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545713    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545715    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545718    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545721    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545723    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545726    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545728    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545730    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545733    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545737    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545740    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545743    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545746    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545749    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545751    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545754    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545756    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545759    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545761    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 13:45:14.547061 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545764    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545767    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545770    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545773    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545776    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545779    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545781    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545785    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545789    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545792    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545794    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545797    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545799    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545802    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545805    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545807    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545810    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545812    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545815    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545817    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 13:45:14.547515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.545819    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546493    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546499    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546502    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546505    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546508    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546511    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546514    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546517    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546520    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546522    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546525    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546529    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546531    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546534    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546537    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546540    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546542    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546545    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546548    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 13:45:14.547982 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546551    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546554    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546557    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546560    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546562    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546565    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546567    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546570    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546572    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546575    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546577    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546580    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546582    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546584    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546587    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546590    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546592    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546595    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546597    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 13:45:14.548486 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546600    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546604    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546607    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546609    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546612    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546614    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546617    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546619    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546621    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546624    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546626    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546629    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546632    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546634    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546637    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546640    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546644    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546647    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546649    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546652    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 13:45:14.548937 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546655    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546657    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546659    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546662    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546664    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546667    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546669    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546671    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546675    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546678    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546681    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546685    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546687    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546691    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546694    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546697    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546700    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546702    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546705    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 13:45:14.549533 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546707    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546709    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546712    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546714    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546717    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546719    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546722    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546725    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546727    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546730    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546732    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546735    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546737    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546740    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546742    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546745    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546747    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546750    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546752    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546754    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 13:45:14.550029 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546757    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.546759    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547116    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547121    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547124    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547127    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547130    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547133    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547135    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547138    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547140    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547143    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547145    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547148    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547150    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547152    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547155    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547160    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547164    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547166    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 13:45:14.550516 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547169    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547173    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547176    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547179    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547182    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547185    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547188    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547190    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547193    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547196    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547199    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547202    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547205    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547208    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547210    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547213    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547215    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547218    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547221    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 13:45:14.550992 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547224    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547226    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547229    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547231    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547234    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547236    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547239    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547242    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547244    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547247    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547251    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547254    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547258    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547261    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547264    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547266    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547268    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547271    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547273    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547276    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 13:45:14.551502 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547278    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547281    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547283    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547286    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547288    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547290    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547292    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547295    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547297    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547300    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547302    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547305    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547307    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547310    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547312    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547316    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547318    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547320    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547323    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547325    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 13:45:14.551989 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547327    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547329    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547333    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547336    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547338    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547341    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547343    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547346    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547348    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547350    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547353    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547355    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547357    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547360    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547363    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547365    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547367    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547369    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547372    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547374    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 13:45:14.552483 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547377    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.547379    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547444    2561 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547450    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547456    2561 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547460    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547466    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547471    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547477    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547483    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547488    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547493    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547498    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547502    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547507    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547510    2561 flags.go:64] FLAG: --cgroup-root=""
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547512    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547515    2561 flags.go:64] FLAG: --client-ca-file=""
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547518    2561 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547521    2561 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547527    2561 flags.go:64] FLAG: --cluster-domain=""
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547530    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547533    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547536    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 13:45:14.552987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547539    2561 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547542    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547545    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547548    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547551    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547554    2561 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547557    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547560    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547562    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547565    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547569    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547572    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547574    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547577    2561 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547581    2561 flags.go:64] FLAG: --enable-server="true"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547584    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547588    2561 flags.go:64] FLAG: --event-burst="100"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547591    2561 flags.go:64] FLAG: --event-qps="50"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547594    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547597    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547599    2561 flags.go:64] FLAG: --eviction-hard=""
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547603    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547606    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547609    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547612    2561 flags.go:64] FLAG: --eviction-soft=""
Jul 17 13:45:14.553582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547614    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547617    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547620    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547623    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547626    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547629    2561 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547631    2561 flags.go:64] FLAG: --feature-gates=""
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547635    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547638    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547640    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547643    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547646    2561 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547649    2561 flags.go:64] FLAG: --help="false"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547651    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-137-163.ec2.internal"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547655    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547657    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547660    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547663    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547667    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547670    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547672    2561 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547676    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547678    2561 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 13:45:14.554414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547682    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547685    2561 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547687    2561 flags.go:64] FLAG: --kube-reserved=""
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547690    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547693    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547696    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547698    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547701    2561 flags.go:64] FLAG: --lock-file=""
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547704    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547706    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547709    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547714    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547717    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547720    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547722    2561 flags.go:64] FLAG: --logging-format="text"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547725    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547728    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547731    2561 flags.go:64] FLAG: --manifest-url=""
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547733    2561 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547737    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547740    2561 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547744    2561 flags.go:64] FLAG: --max-pods="110"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547746    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547749    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547752    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 13:45:14.555418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547755    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547757    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547760    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547763    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547770    2561 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547774    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547777    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547780    2561 flags.go:64] FLAG: --pod-cidr=""
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547790    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547795    2561 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547798    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547801    2561 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547803    2561 flags.go:64] FLAG: --port="10250"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547807    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547810    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-076d773402e72649d"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547813    2561 flags.go:64] FLAG: --qos-reserved=""
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547816    2561 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547819    2561 flags.go:64] FLAG: --register-node="true"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547822    2561 flags.go:64] FLAG: --register-with-taints=""
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547829    2561 flags.go:64] FLAG: --registry-burst="10"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547832    2561 flags.go:64] FLAG: --registry-qps="5"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547835    2561 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547837    2561 flags.go:64] FLAG: --reserved-memory=""
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547841    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547843    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 13:45:14.556334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547846    2561 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547849    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547852    2561 flags.go:64] FLAG: --runonce="false"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547854    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547857    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547860    2561 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547863    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547865    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547868    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547871    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547874    2561 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547876    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547879    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547883    2561 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547886    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547889    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547892    2561 flags.go:64] FLAG: --system-cgroups=""
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547895    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547901    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547903    2561 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547906    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547910    2561 flags.go:64] FLAG: --tls-min-version=""
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547913    2561 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547916    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547918    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 13:45:14.556994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547921    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547924    2561 flags.go:64] FLAG: --v="2"
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547927    2561 flags.go:64] FLAG: --version="false"
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547931    2561 flags.go:64] FLAG: --vmodule=""
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547934    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.547937    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548063    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548068    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548071    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548074    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548077    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548079    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548083    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548085    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548088    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548090    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548093    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548096    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548098    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548101    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548104    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 13:45:14.558185 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548107    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548109    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548112    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548115    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548119    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548125    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548128    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548131    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548134    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548136    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548139    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548142    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548144    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548147    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548149    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548152    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548155    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548157    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548159    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 13:45:14.558773 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548162    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548165    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548167    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548170    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548172    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548174    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548177    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548180    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548182    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548185    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548187    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548190    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548194    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548197    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548199    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548202    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548204    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548207    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548211    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548213    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 13:45:14.559401 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548216    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548219    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548221    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548223    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548226    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548228    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548231    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548233    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548236    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548238    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548241    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548244    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548247    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548250    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548252    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548255    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548257    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548260    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548262    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 13:45:14.559956 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548264    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548267    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548269    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548271    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548275    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548279    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548282    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548285    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548288    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548291    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548293    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548297    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548300    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548302    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548305    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548307    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548310    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548312    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548315    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 13:45:14.560744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548317    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548319    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548322    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548324    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548327    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548329    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.548331    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.549142    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.556060    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.556077    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.556895    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.556911    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.556993    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557021    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557028    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 13:45:14.561329 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557033    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557037    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557041    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557045    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557049    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557053    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557058    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557062    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557066    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557070    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557074    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557078    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557082    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557086    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557090    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557094    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557098    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557102    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557107    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557111    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 13:45:14.561807 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557115    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557119    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557123    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557127    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557131    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557135    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557141    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557146    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557150    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557154    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557158    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557162    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557166    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557171    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557175    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557179    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557184    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557190    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557196    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 13:45:14.562592 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557201    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557206    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557210    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557214    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557218    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557223    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557228    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557232    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557235    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557240    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557244    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557248    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557252    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557256    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557260    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557265    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557270    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557274    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557280    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 13:45:14.563279 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557287    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557302    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557309    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557314    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557320    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557325    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557329    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557334    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557338    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557342    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557346    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557350    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557354    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557358    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557362    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557366    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557370    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557374    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557378    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 13:45:14.563739 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557382    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557386    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557390    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557394    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557398    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557402    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557406    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557410    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557414    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557418    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557422    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557426    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557430    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557435    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557439    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557444    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557448    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557451    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 13:45:14.564238 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557456    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.557464    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557639    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557647    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557652    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557657    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557661    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557665    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557669    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557673    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557677    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557681    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557685    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557689    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557694    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 13:45:14.564678 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557698    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557702    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557706    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557710    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557714    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557718    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557723    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557727    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557731    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557735    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557739    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557743    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557748    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557753    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557757    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557761    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557765    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557769    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557773    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 13:45:14.565058 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557777    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557781    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557785    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557789    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557793    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557797    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557801    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557805    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557810    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557814    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557817    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557821    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557828    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557834    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557839    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557844    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557848    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557852    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557856    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 13:45:14.565520 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557860    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557865    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557869    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557873    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557877    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557881    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557885    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557889    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557894    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557898    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557902    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557906    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557910    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557916    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557922    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557926    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557930    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557934    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557938    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557942    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 13:45:14.565966 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557946    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557950    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557954    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557958    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557966    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557971    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557975    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557980    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557984    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557988    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557992    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.557996    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558000    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558025    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558029    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558032    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558037    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558041    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558045    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558049    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 13:45:14.566453 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558053    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558058    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558061    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558066    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558070    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558074    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558078    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:14.558082    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.558090    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.558222    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.561388    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.564087    2561 server.go:1019] "Starting client certificate rotation"
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.564236    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 13:45:14.566930 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.564296    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 13:45:14.567951 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.567935    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 13:45:14.572752 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.572737    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 13:45:14.581654 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.581637    2561 log.go:25] "Validated CRI v1 runtime API"
Jul 17 13:45:14.583486 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.583473    2561 log.go:25] "Validated CRI v1 image API"
Jul 17 13:45:14.584394 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.584380    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 13:45:14.589744 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.589715    2561 fs.go:135] Filesystem UUIDs: map[3805d019-7b9e-45f4-87b3-c0634cb7c336:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f8047ea9-53a8-40e6-8ff3-c5088cb9663d:/dev/nvme0n1p3]
Jul 17 13:45:14.589801 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.589740    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 13:45:14.594953 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.594835    2561 manager.go:216] Machine: {Timestamp:2026-07-17 13:45:14.593616004 +0000 UTC m=+0.300062829 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102334 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec207f7a5b68dd07667daa6e49d70257 SystemUUID:ec207f7a-5b68-dd07-667d-aa6e49d70257 BootID:3f1dd12b-c540-4d59-b21b-5e353e338a8c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:25:4a:45:71:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:25:4a:45:71:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:45:23:66:5c:be Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 13:45:14.594953 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.594950    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 13:45:14.595068 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.595038    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 13:45:14.597055 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.597031    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 13:45:14.597176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.597056    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-163.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 13:45:14.598720 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.598707    2561 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 13:45:14.598720 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.598721    2561 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 13:45:14.598809 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.598728    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 13:45:14.598809 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.598743    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 13:45:14.599545 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.599521    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 13:45:14.599882 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.599868    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 13:45:14.603073 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.603060    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 13:45:14.603242 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.603230    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 13:45:14.603314 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.603269    2561 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 13:45:14.603314 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.603288    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 13:45:14.603314 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.603308    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 13:45:14.603435 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.603322    2561 kubelet.go:398] "Adding apiserver pod source"
Jul 17 13:45:14.603435 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.603334    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 13:45:14.603731 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.603717    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 13:45:14.603810 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.603742    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 13:45:14.607135 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.606408    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 13:45:14.608302 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.608287    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 13:45:14.608519 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.608502    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 13:45:14.609519 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609505    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 13:45:14.609552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609526    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 13:45:14.609552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609533    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 13:45:14.609552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609538    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 13:45:14.609552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609547    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 13:45:14.609552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609552    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 13:45:14.609675 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609562    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 13:45:14.609675 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609570    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 13:45:14.609675 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609578    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 13:45:14.609675 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609583    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 13:45:14.609675 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609591    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 13:45:14.609675 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.609596    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 13:45:14.610325 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.610314    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 13:45:14.610355 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.610325    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 13:45:14.613635 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.613621    2561 server.go:1268] "Started kubelet"
Jul 17 13:45:14.613770 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.613719    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 13:45:14.614415 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.614346    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 13:45:14.614513 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.614498    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 13:45:14.614512 ip-10-0-137-163 systemd[1]: Started Kubernetes Kubelet.
Jul 17 13:45:14.614887 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.614856    2561 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 13:45:14.615719 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.615703    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 13:45:14.617326 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.617304    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-163.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 13:45:14.617432 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.617352    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 13:45:14.617432 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.617352    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 13:45:14.618139 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.618121    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 13:45:14.620610 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.620595    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 13:45:14.620610 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.620607    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 13:45:14.620733 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.620714    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:14.621257 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621237    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 13:45:14.621346 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621241    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 13:45:14.621346 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621310    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 13:45:14.621498 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621485    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 13:45:14.621539 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621501    2561 factory.go:55] Registering systemd factory
Jul 17 13:45:14.621539 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621469    2561 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 13:45:14.621539 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621509    2561 factory.go:223] Registration of the systemd container factory successfully
Jul 17 13:45:14.621539 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621516    2561 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 13:45:14.621777 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621734    2561 factory.go:153] Registering CRI-O factory
Jul 17 13:45:14.621777 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621747    2561 factory.go:223] Registration of the crio container factory successfully
Jul 17 13:45:14.621777 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621772    2561 factory.go:103] Registering Raw factory
Jul 17 13:45:14.621903 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.621790    2561 manager.go:1196] Started watching for new ooms in manager
Jul 17 13:45:14.622360 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.622348    2561 manager.go:349] Starting recovery of all containers
Jul 17 13:45:14.628212 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.628187    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sdtl4"
Jul 17 13:45:14.628537 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.628515    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 17 13:45:14.629401 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.629351    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 17 13:45:14.630541 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.629244    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-163.ec2.internal.18c317ae44b952a5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-163.ec2.internal,UID:ip-10-0-137-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-163.ec2.internal,},FirstTimestamp:2026-07-17 13:45:14.613592741 +0000 UTC m=+0.320039565,LastTimestamp:2026-07-17 13:45:14.613592741 +0000 UTC m=+0.320039565,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-163.ec2.internal,}"
Jul 17 13:45:14.633209 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.633191    2561 manager.go:354] Recovery completed
Jul 17 13:45:14.637393 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.637240    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sdtl4"
Jul 17 13:45:14.639173 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.639157    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 13:45:14.642479 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.642464    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 13:45:14.642546 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.642492    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 13:45:14.642546 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.642502    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientPID"
Jul 17 13:45:14.643066 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.643052    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 13:45:14.643066 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.643065    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 13:45:14.643163 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.643066    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 13:45:14.643163 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.643132    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 13:45:14.643824 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.643808    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 13:45:14.643886 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.643837    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 13:45:14.643886 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.643846    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientPID"
Jul 17 13:45:14.644072 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.644060    2561 policy_none.go:49] "None policy: Start"
Jul 17 13:45:14.644107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.644077    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 13:45:14.644107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.644088    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 13:45:14.645035 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.645023    2561 policy_none.go:47] "Start"
Jul 17 13:45:14.646494 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.644705    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-163.ec2.internal.18c317ae467218ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-163.ec2.internal,UID:ip-10-0-137-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-163.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-163.ec2.internal,},FirstTimestamp:2026-07-17 13:45:14.64247931 +0000 UTC m=+0.348926134,LastTimestamp:2026-07-17 13:45:14.64247931 +0000 UTC m=+0.348926134,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-163.ec2.internal,}"
Jul 17 13:45:14.682096 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.682084    2561 manager.go:337] "Starting Device Plugin manager"
Jul 17 13:45:14.682152 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.682116    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 13:45:14.682152 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.682126    2561 server.go:87] "Starting device plugin registration server"
Jul 17 13:45:14.682330 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.682316    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 13:45:14.682421 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.682328    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 13:45:14.682469 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.682409    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 13:45:14.682510 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.682501    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 13:45:14.682510 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.682509    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 13:45:14.682976 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.682958    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 13:45:14.683067 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.683000    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:14.763735 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.763707    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 13:45:14.764989 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.764973    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 13:45:14.765087 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.765022    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 13:45:14.765087 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.765042    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 13:45:14.765087 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.765070    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 13:45:14.767452 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.767435    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 13:45:14.783422 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.783407    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 13:45:14.784922 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.784902    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 13:45:14.785038 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.784939    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 13:45:14.785038 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.784954    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientPID"
Jul 17 13:45:14.785038 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.784979    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-163.ec2.internal"
Jul 17 13:45:14.793122 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.793076    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-163.ec2.internal"
Jul 17 13:45:14.793122 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.793100    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-163.ec2.internal\": node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:14.809208 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.809185    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:14.865899 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.865879    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal"]
Jul 17 13:45:14.865983 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.865947    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 13:45:14.866760 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.866745    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 13:45:14.866848 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.866772    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 13:45:14.866848 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.866781    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientPID"
Jul 17 13:45:14.867865 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.867852    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 13:45:14.867985 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.867961    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:14.868071 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.867998    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 13:45:14.868546 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.868533    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 13:45:14.868619 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.868558    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 13:45:14.868619 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.868537    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 13:45:14.868619 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.868583    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 13:45:14.868619 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.868594    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientPID"
Jul 17 13:45:14.868795 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.868568    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientPID"
Jul 17 13:45:14.869721 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.869705    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:14.869805 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.869740    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 13:45:14.870378 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.870363    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 13:45:14.870449 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.870384    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 13:45:14.870449 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.870393    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeHasSufficientPID"
Jul 17 13:45:14.894558 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.894544    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-163.ec2.internal\" not found" node="ip-10-0-137-163.ec2.internal"
Jul 17 13:45:14.898792 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.898779    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-163.ec2.internal\" not found" node="ip-10-0-137-163.ec2.internal"
Jul 17 13:45:14.910111 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:14.910093    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:14.922638 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.922620    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98840faf737d2cdde86a6e7fcaa940d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal\" (UID: \"98840faf737d2cdde86a6e7fcaa940d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:14.922693 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.922643    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ad89fec431916698e96755d7734948e7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-163.ec2.internal\" (UID: \"ad89fec431916698e96755d7734948e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:14.922693 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:14.922669    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/98840faf737d2cdde86a6e7fcaa940d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal\" (UID: \"98840faf737d2cdde86a6e7fcaa940d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:15.010343 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:15.010320    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:15.023724 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.023706    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98840faf737d2cdde86a6e7fcaa940d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal\" (UID: \"98840faf737d2cdde86a6e7fcaa940d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:15.023777 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.023729    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ad89fec431916698e96755d7734948e7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-163.ec2.internal\" (UID: \"ad89fec431916698e96755d7734948e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:15.023777 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.023751    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/98840faf737d2cdde86a6e7fcaa940d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal\" (UID: \"98840faf737d2cdde86a6e7fcaa940d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:15.023777 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.023770    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98840faf737d2cdde86a6e7fcaa940d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal\" (UID: \"98840faf737d2cdde86a6e7fcaa940d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:15.023868 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.023774    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/98840faf737d2cdde86a6e7fcaa940d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal\" (UID: \"98840faf737d2cdde86a6e7fcaa940d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:15.023868 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.023772    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ad89fec431916698e96755d7734948e7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-163.ec2.internal\" (UID: \"ad89fec431916698e96755d7734948e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:15.110677 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:15.110613    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:15.197725 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.197703    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:15.201767 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.201455    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:15.211323 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:15.211303    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:15.311748 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:15.311726    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:15.412303 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:15.412261    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:15.512716 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:15.512695    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:15.563098 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.563078    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 13:45:15.563444 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.563197    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 13:45:15.613545 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:15.613527    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:15.640760 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.640738    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 13:40:15 +0000 UTC" deadline="2028-02-14 12:30:37.203871904 +0000 UTC"
Jul 17 13:45:15.640760 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.640760    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13846h45m21.563114412s"
Jul 17 13:45:15.650462 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.650446    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 13:45:15.714630 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:15.714581    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:15.815357 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:15.815327    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:15.829515 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:15.829490    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98840faf737d2cdde86a6e7fcaa940d5.slice/crio-a9ed3b161e18516fef7cae60d3c80166409c0078a6651f35491efb0720e16e68 WatchSource:0}: Error finding container a9ed3b161e18516fef7cae60d3c80166409c0078a6651f35491efb0720e16e68: Status 404 returned error can't find the container with id a9ed3b161e18516fef7cae60d3c80166409c0078a6651f35491efb0720e16e68
Jul 17 13:45:15.835969 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:15.835954    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 13:45:15.915572 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:15.915547    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:16.017629 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.017565    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:16.070189 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:16.070160    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad89fec431916698e96755d7734948e7.slice/crio-d25c55dc6416b2ca3a01c6a33fa53d74fc1ea3c7a43fdf0b4d24b535df3108d8 WatchSource:0}: Error finding container d25c55dc6416b2ca3a01c6a33fa53d74fc1ea3c7a43fdf0b4d24b535df3108d8: Status 404 returned error can't find the container with id d25c55dc6416b2ca3a01c6a33fa53d74fc1ea3c7a43fdf0b4d24b535df3108d8
Jul 17 13:45:16.117946 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.117913    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-163.ec2.internal\" not found"
Jul 17 13:45:16.185337 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.185309    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 13:45:16.206313 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.206293    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 13:45:16.221375 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.221347    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:16.237941 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.237925    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 13:45:16.238883 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.238870    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal"
Jul 17 13:45:16.251173 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.251159    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 13:45:16.604957 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.604928    2561 apiserver.go:52] "Watching apiserver"
Jul 17 13:45:16.614282 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.613674    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 13:45:16.616730 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.616683    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-gsr4m","kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg","openshift-dns/node-resolver-z77wd","openshift-image-registry/node-ca-xtx5f","openshift-network-diagnostics/network-check-target-8gtz9","openshift-ovn-kubernetes/ovnkube-node-s56n4","kube-system/konnectivity-agent-dztc6","openshift-cluster-node-tuning-operator/tuned-r7x4d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal","openshift-multus/multus-additional-cni-plugins-dfkch","openshift-multus/multus-mpv6f","openshift-multus/network-metrics-daemon-7dbv7","openshift-network-operator/iptables-alerter-99rv5"]
Jul 17 13:45:16.620018 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.619993    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.624649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.621999    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 13:45:16.624649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.623051    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 13:45:16.624649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.623249    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 13:45:16.624649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.623876    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:16.624649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.623892    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 13:45:16.624649 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.623932    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:16.624649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.624185    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 13:45:16.624649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.624260    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 13:45:16.624649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.624526    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 13:45:16.626639 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.625856    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.629449 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.628063    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fvrtp\""
Jul 17 13:45:16.629449 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.628301    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.630350 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.629910    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 13:45:16.630657 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.630636    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.631450 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631397    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-cni-bin\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.631450 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631430    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-run-netns\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.631617 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631457    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-etc-openvswitch\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.631617 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631480    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-node-log\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.631617 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631504    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed26acdd-2775-46d9-b179-17def1c1d2f9-ovn-node-metrics-cert\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.631617 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631531    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqpcg\" (UniqueName: \"kubernetes.io/projected/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-kube-api-access-mqpcg\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:16.631617 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631554    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-etc-selinux\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.631617 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631578    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-sys-fs\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.631617 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631611    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-slash\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631634    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed26acdd-2775-46d9-b179-17def1c1d2f9-env-overrides\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631657    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-kubelet\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631682    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-var-lib-openvswitch\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631708    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631731    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-cni-netd\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631758    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-socket-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631789    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-registration-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631812    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-systemd-units\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631850    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed26acdd-2775-46d9-b179-17def1c1d2f9-ovnkube-config\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631876    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631900    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-device-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631924    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-log-socket\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631948    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631975    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jndn4\" (UniqueName: \"kubernetes.io/projected/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-kube-api-access-jndn4\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.632042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.631999    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-run-openvswitch\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632876 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.632049    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed26acdd-2775-46d9-b179-17def1c1d2f9-ovnkube-script-lib\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632876 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.632074    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-run-systemd\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632876 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.632131    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-run-ovn\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632876 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.632158    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.632876 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.632220    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkt8w\" (UniqueName: \"kubernetes.io/projected/ed26acdd-2775-46d9-b179-17def1c1d2f9-kube-api-access-dkt8w\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.634001 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.633464    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:16.634001 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.633684    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:16.635172 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.634405    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 13:45:16.635172 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.634426    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 13:45:16.635172 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.634435    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 13:45:16.635172 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.634664    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 13:45:16.635172 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.634681    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 13:45:16.635172 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.634846    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wjsrq\""
Jul 17 13:45:16.635172 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.634898    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 13:45:16.635172 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.635041    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 13:45:16.635689 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.635210    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xd7qw\""
Jul 17 13:45:16.635689 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.635313    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kvc82\""
Jul 17 13:45:16.636215 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.636193    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.639063 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.639038    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:16.639191 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.639112    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:16.641209 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.641187    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 13:45:16.641455 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.641437    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cvf66\""
Jul 17 13:45:16.641860 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.641824    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.644869 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.644835    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:16.646476 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.646453    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 13:45:16.646972 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.646954    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.647211 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.647160    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 13:45:16.647447 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.647417    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 13:45:16.647919 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.647899    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 13:45:16.648466 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.648440    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 13:45:16.648670 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.648649    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 13:45:16.648861 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.648844    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 13:45:16.649082 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.649041    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9l7m4\""
Jul 17 13:45:16.649217 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.649196    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mkr9q\""
Jul 17 13:45:16.649887 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.649859    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.654988 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.653462    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 13:45:16.654988 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.653889    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 13:45:16.654988 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.653899    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-94xm6\""
Jul 17 13:45:16.654988 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.654181    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 13:45:16.655600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.655572    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 13:45:16.655754 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.655735    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 13:45:16.657342 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.657310    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qqgc7\""
Jul 17 13:45:16.657893 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.657871    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 13:45:16.722668 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.722646    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 13:45:16.734239 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734214    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-device-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.734329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734256    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-log-socket\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.734329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734290    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0825032-81b7-4c8c-8494-cab86102a66c-serviceca\") pod \"node-ca-xtx5f\" (UID: \"b0825032-81b7-4c8c-8494-cab86102a66c\") " pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.734329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734314    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-sys\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.734479 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734341    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed26acdd-2775-46d9-b179-17def1c1d2f9-ovnkube-script-lib\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.734479 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734366    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f83860e4-2ac5-4187-aa2f-45fcc803faf5-kubelet-config\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:16.734479 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734393    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-cnibin\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.734479 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734418    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/21b1cb43-5f8c-4bc6-9452-b6f39aa68e71-agent-certs\") pod \"konnectivity-agent-dztc6\" (UID: \"21b1cb43-5f8c-4bc6-9452-b6f39aa68e71\") " pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:16.734479 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734454    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.734639 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734481    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkt8w\" (UniqueName: \"kubernetes.io/projected/ed26acdd-2775-46d9-b179-17def1c1d2f9-kube-api-access-dkt8w\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.734639 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734506    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-run-k8s-cni-cncf-io\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.734639 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734532    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-run-netns\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.734639 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734558    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-cni-bin\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.734639 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734581    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-sysctl-d\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.734639 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734606    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-systemd\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.734639 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734635    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-daemon-config\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.734870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734661    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-run-multus-certs\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.734870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734690    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2sf7\" (UniqueName: \"kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7\") pod \"network-check-target-8gtz9\" (UID: \"f3ba7951-5c0a-4d39-8c81-a325d1abf091\") " pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:16.734870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734716    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-etc-openvswitch\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.734870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734741    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-node-log\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.734870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734766    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed26acdd-2775-46d9-b179-17def1c1d2f9-ovn-node-metrics-cert\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.734870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734790    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-etc-kubernetes\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.734870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734816    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqpcg\" (UniqueName: \"kubernetes.io/projected/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-kube-api-access-mqpcg\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:16.734870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734844    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-sys-fs\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.734870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734869    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-slash\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.735176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734893    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.735176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734926    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-cni-netd\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.735176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734958    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:16.735176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.734991    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.735176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735016    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-var-lib-kubelet\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.735176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735046    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-socket-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.735176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735072    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-systemd-units\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.735176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735122    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed26acdd-2775-46d9-b179-17def1c1d2f9-ovnkube-config\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.735176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735157    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drtrv\" (UniqueName: \"kubernetes.io/projected/e1ae684d-4434-45b3-8075-749da32e5be8-kube-api-access-drtrv\") pod \"iptables-alerter-99rv5\" (UID: \"e1ae684d-4434-45b3-8075-749da32e5be8\") " pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.735457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735190    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-system-cni-dir\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.735457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735237    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdzw9\" (UniqueName: \"kubernetes.io/projected/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-kube-api-access-qdzw9\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.735457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735270    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-sysconfig\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.735457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735296    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.735457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735330    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e1ae684d-4434-45b3-8075-749da32e5be8-iptables-alerter-script\") pod \"iptables-alerter-99rv5\" (UID: \"e1ae684d-4434-45b3-8075-749da32e5be8\") " pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.735457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735357    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.735457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735386    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-kubernetes\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.735457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735411    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-tuned\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.735457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735435    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-socket-dir-parent\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.735726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735460    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-hostroot\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.735726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735484    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfvn7\" (UniqueName: \"kubernetes.io/projected/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-kube-api-access-wfvn7\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.735726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735514    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:16.735726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735542    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jndn4\" (UniqueName: \"kubernetes.io/projected/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-kube-api-access-jndn4\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.735726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735570    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-run-openvswitch\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.735726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735607    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-lib-modules\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.735726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735635    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-cnibin\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.735726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735659    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-os-release\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.735726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735685    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-run-systemd\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.735726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735710    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-run-ovn\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735736    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-modprobe-d\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735764    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-var-lib-cni-multus\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735794    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-conf-dir\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735824    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-os-release\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735850    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-system-cni-dir\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735878    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-run-netns\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735906    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f222c7e6-2c4d-402a-879c-4a0ca6ae2475-hosts-file\") pod \"node-resolver-z77wd\" (UID: \"f222c7e6-2c4d-402a-879c-4a0ca6ae2475\") " pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735934    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f222c7e6-2c4d-402a-879c-4a0ca6ae2475-tmp-dir\") pod \"node-resolver-z77wd\" (UID: \"f222c7e6-2c4d-402a-879c-4a0ca6ae2475\") " pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735960    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzdf9\" (UniqueName: \"kubernetes.io/projected/b0825032-81b7-4c8c-8494-cab86102a66c-kube-api-access-bzdf9\") pod \"node-ca-xtx5f\" (UID: \"b0825032-81b7-4c8c-8494-cab86102a66c\") " pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.735988    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f83860e4-2ac5-4187-aa2f-45fcc803faf5-dbus\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:16.736015 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736014    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-cni-binary-copy\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736039    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-host\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736065    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/21b1cb43-5f8c-4bc6-9452-b6f39aa68e71-konnectivity-ca\") pod \"konnectivity-agent-dztc6\" (UID: \"21b1cb43-5f8c-4bc6-9452-b6f39aa68e71\") " pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736111    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-etc-selinux\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736139    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed26acdd-2775-46d9-b179-17def1c1d2f9-env-overrides\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736172    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88xcr\" (UniqueName: \"kubernetes.io/projected/f222c7e6-2c4d-402a-879c-4a0ca6ae2475-kube-api-access-88xcr\") pod \"node-resolver-z77wd\" (UID: \"f222c7e6-2c4d-402a-879c-4a0ca6ae2475\") " pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736203    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1ae684d-4434-45b3-8075-749da32e5be8-host-slash\") pod \"iptables-alerter-99rv5\" (UID: \"e1ae684d-4434-45b3-8075-749da32e5be8\") " pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736227    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b21695c3-88ac-45c2-bcce-e1a6bb654650-tmp\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736253    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp47c\" (UniqueName: \"kubernetes.io/projected/b21695c3-88ac-45c2-bcce-e1a6bb654650-kube-api-access-xp47c\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736281    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-cni-dir\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736307    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-cni-binary-copy\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736334    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-kubelet\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736362    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-var-lib-openvswitch\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736387    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0825032-81b7-4c8c-8494-cab86102a66c-host\") pod \"node-ca-xtx5f\" (UID: \"b0825032-81b7-4c8c-8494-cab86102a66c\") " pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736414    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736446    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-registration-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.736485 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736471    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-sysctl-conf\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.737305 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736498    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-run\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.737305 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736523    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-var-lib-kubelet\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.737305 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736549    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-var-lib-cni-bin\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.737305 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736658    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-device-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.737305 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.736711    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-log-socket\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.737305 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.737277    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed26acdd-2775-46d9-b179-17def1c1d2f9-ovnkube-script-lib\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.737599 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.737405    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.737666    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-run-openvswitch\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.737757    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-run-systemd\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.737805    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-run-ovn\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.737912    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-cni-bin\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.738057    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-etc-openvswitch\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.738123    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-node-log\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.738491    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.738523    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-sys-fs\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739040    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-socket-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739236    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-etc-selinux\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739484    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739492    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-kubelet\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739561    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-slash\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739585    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-cni-netd\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739607    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-var-lib-openvswitch\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739612    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed26acdd-2775-46d9-b179-17def1c1d2f9-env-overrides\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739682    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.740552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-host-run-netns\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.741546 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739770    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-registration-dir\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.741546 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.739815    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed26acdd-2775-46d9-b179-17def1c1d2f9-systemd-units\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.741546 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.740037    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed26acdd-2775-46d9-b179-17def1c1d2f9-ovnkube-config\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.741546 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.740140    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:16.741546 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.740212    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs podName:c5ffda97-3c84-479c-b0c2-d1c62ddc90d7 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:17.240190236 +0000 UTC m=+2.946637052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs") pod "network-metrics-daemon-7dbv7" (UID: "c5ffda97-3c84-479c-b0c2-d1c62ddc90d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:16.742477 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.742453    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed26acdd-2775-46d9-b179-17def1c1d2f9-ovn-node-metrics-cert\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.745914 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.745895    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4q57r"
Jul 17 13:45:16.755721 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.755699    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqpcg\" (UniqueName: \"kubernetes.io/projected/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-kube-api-access-mqpcg\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:16.757068 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.757028    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkt8w\" (UniqueName: \"kubernetes.io/projected/ed26acdd-2775-46d9-b179-17def1c1d2f9-kube-api-access-dkt8w\") pod \"ovnkube-node-s56n4\" (UID: \"ed26acdd-2775-46d9-b179-17def1c1d2f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.757597 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.757568    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jndn4\" (UniqueName: \"kubernetes.io/projected/0f0af0eb-6443-4fb4-8137-e0179c0ab7c6-kube-api-access-jndn4\") pod \"aws-ebs-csi-driver-node-b6hlg\" (UID: \"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.774343 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.768610    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4q57r"
Jul 17 13:45:16.774343 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.772583    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal" event={"ID":"ad89fec431916698e96755d7734948e7","Type":"ContainerStarted","Data":"d25c55dc6416b2ca3a01c6a33fa53d74fc1ea3c7a43fdf0b4d24b535df3108d8"}
Jul 17 13:45:16.774343 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.773971    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal" event={"ID":"98840faf737d2cdde86a6e7fcaa940d5","Type":"ContainerStarted","Data":"a9ed3b161e18516fef7cae60d3c80166409c0078a6651f35491efb0720e16e68"}
Jul 17 13:45:16.837114 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837082    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-sysctl-conf\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.837294 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837123    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-run\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.837294 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837158    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-var-lib-kubelet\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.837294 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837193    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-var-lib-cni-bin\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.837294 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0825032-81b7-4c8c-8494-cab86102a66c-serviceca\") pod \"node-ca-xtx5f\" (UID: \"b0825032-81b7-4c8c-8494-cab86102a66c\") " pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.837294 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837277    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-sys\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.837294 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837279    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-sysctl-conf\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837301    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f83860e4-2ac5-4187-aa2f-45fcc803faf5-kubelet-config\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837348    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-var-lib-cni-bin\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837336    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-cnibin\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837382    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/21b1cb43-5f8c-4bc6-9452-b6f39aa68e71-agent-certs\") pod \"konnectivity-agent-dztc6\" (UID: \"21b1cb43-5f8c-4bc6-9452-b6f39aa68e71\") " pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837396    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-run\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837409    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-run-k8s-cni-cncf-io\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837445    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-var-lib-kubelet\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837447    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-run-netns\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837491    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-sysctl-d\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837498    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-run-netns\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837515    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-systemd\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837540    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-daemon-config\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.837573 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837566    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-run-multus-certs\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837594    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sf7\" (UniqueName: \"kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7\") pod \"network-check-target-8gtz9\" (UID: \"f3ba7951-5c0a-4d39-8c81-a325d1abf091\") " pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-etc-kubernetes\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837653    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837677    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837702    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-var-lib-kubelet\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837732    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drtrv\" (UniqueName: \"kubernetes.io/projected/e1ae684d-4434-45b3-8075-749da32e5be8-kube-api-access-drtrv\") pod \"iptables-alerter-99rv5\" (UID: \"e1ae684d-4434-45b3-8075-749da32e5be8\") " pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837757    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-system-cni-dir\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837782    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdzw9\" (UniqueName: \"kubernetes.io/projected/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-kube-api-access-qdzw9\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837810    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-sysconfig\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837836    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e1ae684d-4434-45b3-8075-749da32e5be8-iptables-alerter-script\") pod \"iptables-alerter-99rv5\" (UID: \"e1ae684d-4434-45b3-8075-749da32e5be8\") " pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837856    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837881    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-kubernetes\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837897    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-tuned\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837912    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-socket-dir-parent\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837936    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-hostroot\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837956    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfvn7\" (UniqueName: \"kubernetes.io/projected/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-kube-api-access-wfvn7\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.838158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837958    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0825032-81b7-4c8c-8494-cab86102a66c-serviceca\") pod \"node-ca-xtx5f\" (UID: \"b0825032-81b7-4c8c-8494-cab86102a66c\") " pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.837985    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-lib-modules\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838002    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-cnibin\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838019    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-os-release\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838031    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-sys\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838055    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-modprobe-d\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838103    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f83860e4-2ac5-4187-aa2f-45fcc803faf5-kubelet-config\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838124    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-run-k8s-cni-cncf-io\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838146    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-etc-kubernetes\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838175    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-var-lib-cni-multus\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838185    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-modprobe-d\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838205    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-conf-dir\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838236    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-sysconfig\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838251    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-os-release\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838255    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-sysctl-d\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838280    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-system-cni-dir\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838300    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-systemd\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838311    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f222c7e6-2c4d-402a-879c-4a0ca6ae2475-hosts-file\") pod \"node-resolver-z77wd\" (UID: \"f222c7e6-2c4d-402a-879c-4a0ca6ae2475\") " pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.839061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838337    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f222c7e6-2c4d-402a-879c-4a0ca6ae2475-tmp-dir\") pod \"node-resolver-z77wd\" (UID: \"f222c7e6-2c4d-402a-879c-4a0ca6ae2475\") " pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838366    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzdf9\" (UniqueName: \"kubernetes.io/projected/b0825032-81b7-4c8c-8494-cab86102a66c-kube-api-access-bzdf9\") pod \"node-ca-xtx5f\" (UID: \"b0825032-81b7-4c8c-8494-cab86102a66c\") " pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838375    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-var-lib-kubelet\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838402    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f83860e4-2ac5-4187-aa2f-45fcc803faf5-dbus\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838431    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-cni-binary-copy\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838460    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-host\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838491    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/21b1cb43-5f8c-4bc6-9452-b6f39aa68e71-konnectivity-ca\") pod \"konnectivity-agent-dztc6\" (UID: \"21b1cb43-5f8c-4bc6-9452-b6f39aa68e71\") " pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838522    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88xcr\" (UniqueName: \"kubernetes.io/projected/f222c7e6-2c4d-402a-879c-4a0ca6ae2475-kube-api-access-88xcr\") pod \"node-resolver-z77wd\" (UID: \"f222c7e6-2c4d-402a-879c-4a0ca6ae2475\") " pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838550    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1ae684d-4434-45b3-8075-749da32e5be8-host-slash\") pod \"iptables-alerter-99rv5\" (UID: \"e1ae684d-4434-45b3-8075-749da32e5be8\") " pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838574    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b21695c3-88ac-45c2-bcce-e1a6bb654650-tmp\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838576    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-hostroot\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838621    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-system-cni-dir\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838665    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp47c\" (UniqueName: \"kubernetes.io/projected/b21695c3-88ac-45c2-bcce-e1a6bb654650-kube-api-access-xp47c\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838725    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-run-multus-certs\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838774    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-cni-dir\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838805    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-cni-binary-copy\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838833    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0825032-81b7-4c8c-8494-cab86102a66c-host\") pod \"node-ca-xtx5f\" (UID: \"b0825032-81b7-4c8c-8494-cab86102a66c\") " pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.839999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838840    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.838861    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839291    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-daemon-config\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839410    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-socket-dir-parent\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839483    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-kubernetes\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839496    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e1ae684d-4434-45b3-8075-749da32e5be8-iptables-alerter-script\") pod \"iptables-alerter-99rv5\" (UID: \"e1ae684d-4434-45b3-8075-749da32e5be8\") " pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839569    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839616    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-lib-modules\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839649    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839731    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-cnibin\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839791    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-os-release\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839834    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0825032-81b7-4c8c-8494-cab86102a66c-host\") pod \"node-ca-xtx5f\" (UID: \"b0825032-81b7-4c8c-8494-cab86102a66c\") " pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839899    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b21695c3-88ac-45c2-bcce-e1a6bb654650-host\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839910    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-system-cni-dir\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839931    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-os-release\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.839994    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f222c7e6-2c4d-402a-879c-4a0ca6ae2475-hosts-file\") pod \"node-resolver-z77wd\" (UID: \"f222c7e6-2c4d-402a-879c-4a0ca6ae2475\") " pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.840026    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f83860e4-2ac5-4187-aa2f-45fcc803faf5-dbus\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.840276    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-cni-binary-copy\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.840880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.840293    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f222c7e6-2c4d-402a-879c-4a0ca6ae2475-tmp-dir\") pod \"node-resolver-z77wd\" (UID: \"f222c7e6-2c4d-402a-879c-4a0ca6ae2475\") " pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.841694 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.840333    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-cni-dir\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.841694 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.840389    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-host-var-lib-cni-multus\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.841694 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.840457    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-multus-conf-dir\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.841694 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.840628    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:16.841694 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.840696    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret podName:f83860e4-2ac5-4187-aa2f-45fcc803faf5 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:17.340673491 +0000 UTC m=+3.047120326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret") pod "global-pull-secret-syncer-gsr4m" (UID: "f83860e4-2ac5-4187-aa2f-45fcc803faf5") : object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:16.841694 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.840714    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-cni-binary-copy\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.841694 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.840713    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/21b1cb43-5f8c-4bc6-9452-b6f39aa68e71-konnectivity-ca\") pod \"konnectivity-agent-dztc6\" (UID: \"21b1cb43-5f8c-4bc6-9452-b6f39aa68e71\") " pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:16.841694 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.840783    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1ae684d-4434-45b3-8075-749da32e5be8-host-slash\") pod \"iptables-alerter-99rv5\" (UID: \"e1ae684d-4434-45b3-8075-749da32e5be8\") " pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.841694 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.840899    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-cnibin\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.844117 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.844069    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b21695c3-88ac-45c2-bcce-e1a6bb654650-tmp\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.844671 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.844646    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/21b1cb43-5f8c-4bc6-9452-b6f39aa68e71-agent-certs\") pod \"konnectivity-agent-dztc6\" (UID: \"21b1cb43-5f8c-4bc6-9452-b6f39aa68e71\") " pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:16.845404 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.845382    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b21695c3-88ac-45c2-bcce-e1a6bb654650-etc-tuned\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.874713 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.874638    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdzw9\" (UniqueName: \"kubernetes.io/projected/dea22213-7df3-4fa7-a7c8-052bc5cbabbf-kube-api-access-qdzw9\") pod \"multus-additional-cni-plugins-dfkch\" (UID: \"dea22213-7df3-4fa7-a7c8-052bc5cbabbf\") " pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:16.875667 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.875640    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drtrv\" (UniqueName: \"kubernetes.io/projected/e1ae684d-4434-45b3-8075-749da32e5be8-kube-api-access-drtrv\") pod \"iptables-alerter-99rv5\" (UID: \"e1ae684d-4434-45b3-8075-749da32e5be8\") " pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.879792 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.879771    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88xcr\" (UniqueName: \"kubernetes.io/projected/f222c7e6-2c4d-402a-879c-4a0ca6ae2475-kube-api-access-88xcr\") pod \"node-resolver-z77wd\" (UID: \"f222c7e6-2c4d-402a-879c-4a0ca6ae2475\") " pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.880556 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.880502    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfvn7\" (UniqueName: \"kubernetes.io/projected/5fd17b04-3478-47e5-bd54-66e8e7ee1cef-kube-api-access-wfvn7\") pod \"multus-mpv6f\" (UID: \"5fd17b04-3478-47e5-bd54-66e8e7ee1cef\") " pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:16.881066 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.880801    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 13:45:16.881066 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.880828    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 13:45:16.881066 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.880844    2561 projected.go:196] Error preparing data for projected volume kube-api-access-q2sf7 for pod openshift-network-diagnostics/network-check-target-8gtz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:16.881066 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:16.880900    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7 podName:f3ba7951-5c0a-4d39-8c81-a325d1abf091 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:17.380885113 +0000 UTC m=+3.087331926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q2sf7" (UniqueName: "kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7") pod "network-check-target-8gtz9" (UID: "f3ba7951-5c0a-4d39-8c81-a325d1abf091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:16.882310 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.882265    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzdf9\" (UniqueName: \"kubernetes.io/projected/b0825032-81b7-4c8c-8494-cab86102a66c-kube-api-access-bzdf9\") pod \"node-ca-xtx5f\" (UID: \"b0825032-81b7-4c8c-8494-cab86102a66c\") " pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.883583 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.883562    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp47c\" (UniqueName: \"kubernetes.io/projected/b21695c3-88ac-45c2-bcce-e1a6bb654650-kube-api-access-xp47c\") pod \"tuned-r7x4d\" (UID: \"b21695c3-88ac-45c2-bcce-e1a6bb654650\") " pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.940312 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.940290    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:16.951354 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.951332    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg"
Jul 17 13:45:16.960302 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.960276    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z77wd"
Jul 17 13:45:16.969179 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.969159    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xtx5f"
Jul 17 13:45:16.977929 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.977908    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r7x4d"
Jul 17 13:45:16.981918 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.981885    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 13:45:16.987479 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.987459    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-99rv5"
Jul 17 13:45:16.996222 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:16.996085    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:17.005984 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:17.005960    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dfkch"
Jul 17 13:45:17.014241 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:17.014220    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mpv6f"
Jul 17 13:45:17.241214 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:17.241132    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:17.241354 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:17.241235    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:17.241354 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:17.241289    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs podName:c5ffda97-3c84-479c-b0c2-d1c62ddc90d7 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:18.241272286 +0000 UTC m=+3.947719099 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs") pod "network-metrics-daemon-7dbv7" (UID: "c5ffda97-3c84-479c-b0c2-d1c62ddc90d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:17.341856 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:17.341822    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:17.342038 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:17.341953    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:17.342038 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:17.342023    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret podName:f83860e4-2ac5-4187-aa2f-45fcc803faf5 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:18.342001701 +0000 UTC m=+4.048448518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret") pod "global-pull-secret-syncer-gsr4m" (UID: "f83860e4-2ac5-4187-aa2f-45fcc803faf5") : object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:17.443020 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:17.442982    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sf7\" (UniqueName: \"kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7\") pod \"network-check-target-8gtz9\" (UID: \"f3ba7951-5c0a-4d39-8c81-a325d1abf091\") " pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:17.443192 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:17.443169    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 13:45:17.443192 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:17.443197    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 13:45:17.443362 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:17.443212    2561 projected.go:196] Error preparing data for projected volume kube-api-access-q2sf7 for pod openshift-network-diagnostics/network-check-target-8gtz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:17.443362 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:17.443279    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7 podName:f3ba7951-5c0a-4d39-8c81-a325d1abf091 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:18.443259206 +0000 UTC m=+4.149706022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2sf7" (UniqueName: "kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7") pod "network-check-target-8gtz9" (UID: "f3ba7951-5c0a-4d39-8c81-a325d1abf091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:17.765809 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:17.765778    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:17.766360 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:17.765887    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:17.773909 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:17.773879    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 13:40:17 +0000 UTC" deadline="2028-01-25 19:36:18.221617498 +0000 UTC"
Jul 17 13:45:17.773909 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:17.773907    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13373h51m0.447714467s"
Jul 17 13:45:17.959602 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:17.959577    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb21695c3_88ac_45c2_bcce_e1a6bb654650.slice/crio-f7022abe27c26064be025b24b73aae96073f9f5fbb3ffcb7c55ea2074f46751b WatchSource:0}: Error finding container f7022abe27c26064be025b24b73aae96073f9f5fbb3ffcb7c55ea2074f46751b: Status 404 returned error can't find the container with id f7022abe27c26064be025b24b73aae96073f9f5fbb3ffcb7c55ea2074f46751b
Jul 17 13:45:17.961557 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:17.961530    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0825032_81b7_4c8c_8494_cab86102a66c.slice/crio-7034f59071d16a56f37536769f98b15cfc222e8d843089ea68e5a213f4c66a83 WatchSource:0}: Error finding container 7034f59071d16a56f37536769f98b15cfc222e8d843089ea68e5a213f4c66a83: Status 404 returned error can't find the container with id 7034f59071d16a56f37536769f98b15cfc222e8d843089ea68e5a213f4c66a83
Jul 17 13:45:17.962817 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:17.962774    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddea22213_7df3_4fa7_a7c8_052bc5cbabbf.slice/crio-07ff5c0ccb023bb82a3026ea2879c22d4c08babaa6d1734b6c64d94507fed847 WatchSource:0}: Error finding container 07ff5c0ccb023bb82a3026ea2879c22d4c08babaa6d1734b6c64d94507fed847: Status 404 returned error can't find the container with id 07ff5c0ccb023bb82a3026ea2879c22d4c08babaa6d1734b6c64d94507fed847
Jul 17 13:45:17.966512 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:17.966483    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ae684d_4434_45b3_8075_749da32e5be8.slice/crio-663bb1df3446faabc730f7e38fbfe1c5cd7eea321c4f6c286b78e50b86b16e58 WatchSource:0}: Error finding container 663bb1df3446faabc730f7e38fbfe1c5cd7eea321c4f6c286b78e50b86b16e58: Status 404 returned error can't find the container with id 663bb1df3446faabc730f7e38fbfe1c5cd7eea321c4f6c286b78e50b86b16e58
Jul 17 13:45:17.990206 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:17.990180    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf222c7e6_2c4d_402a_879c_4a0ca6ae2475.slice/crio-208147fd9a86e947e9f802fb5bd770da9ec1acf2bcdb1546a3ec09aaef6a9602 WatchSource:0}: Error finding container 208147fd9a86e947e9f802fb5bd770da9ec1acf2bcdb1546a3ec09aaef6a9602: Status 404 returned error can't find the container with id 208147fd9a86e947e9f802fb5bd770da9ec1acf2bcdb1546a3ec09aaef6a9602
Jul 17 13:45:17.991045 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:17.990971    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0af0eb_6443_4fb4_8137_e0179c0ab7c6.slice/crio-1ed9f1fbf5d85a10f11bf10ab4d00c71ba75000258c4659bf6e6ab215ce4fefd WatchSource:0}: Error finding container 1ed9f1fbf5d85a10f11bf10ab4d00c71ba75000258c4659bf6e6ab215ce4fefd: Status 404 returned error can't find the container with id 1ed9f1fbf5d85a10f11bf10ab4d00c71ba75000258c4659bf6e6ab215ce4fefd
Jul 17 13:45:17.991976 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:17.991924    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fd17b04_3478_47e5_bd54_66e8e7ee1cef.slice/crio-80c7a6531efdfdece33430d5ab3d6befb7452af63f00a7832147a7f7f51bf67a WatchSource:0}: Error finding container 80c7a6531efdfdece33430d5ab3d6befb7452af63f00a7832147a7f7f51bf67a: Status 404 returned error can't find the container with id 80c7a6531efdfdece33430d5ab3d6befb7452af63f00a7832147a7f7f51bf67a
Jul 17 13:45:17.993381 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:17.993325    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b1cb43_5f8c_4bc6_9452_b6f39aa68e71.slice/crio-07bb9ec8dad4efe8727c4788833b41cac71021d6157eda54108254d2da23da87 WatchSource:0}: Error finding container 07bb9ec8dad4efe8727c4788833b41cac71021d6157eda54108254d2da23da87: Status 404 returned error can't find the container with id 07bb9ec8dad4efe8727c4788833b41cac71021d6157eda54108254d2da23da87
Jul 17 13:45:18.251117 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.250906    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:18.251234 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:18.251091    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:18.251318 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:18.251276    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs podName:c5ffda97-3c84-479c-b0c2-d1c62ddc90d7 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:20.251248549 +0000 UTC m=+5.957695385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs") pod "network-metrics-daemon-7dbv7" (UID: "c5ffda97-3c84-479c-b0c2-d1c62ddc90d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:18.351752 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.351636    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:18.351908 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:18.351891    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:18.352028 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:18.352014    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret podName:f83860e4-2ac5-4187-aa2f-45fcc803faf5 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:20.351996448 +0000 UTC m=+6.058443266 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret") pod "global-pull-secret-syncer-gsr4m" (UID: "f83860e4-2ac5-4187-aa2f-45fcc803faf5") : object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:18.453250 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.452596    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sf7\" (UniqueName: \"kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7\") pod \"network-check-target-8gtz9\" (UID: \"f3ba7951-5c0a-4d39-8c81-a325d1abf091\") " pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:18.453250 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:18.452780    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 13:45:18.453250 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:18.452798    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 13:45:18.453250 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:18.452811    2561 projected.go:196] Error preparing data for projected volume kube-api-access-q2sf7 for pod openshift-network-diagnostics/network-check-target-8gtz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:18.453250 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:18.452869    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7 podName:f3ba7951-5c0a-4d39-8c81-a325d1abf091 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:20.452851145 +0000 UTC m=+6.159297965 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2sf7" (UniqueName: "kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7") pod "network-check-target-8gtz9" (UID: "f3ba7951-5c0a-4d39-8c81-a325d1abf091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:18.774618 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.773518    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:18.774618 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:18.773715    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:18.774618 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.774258    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 13:40:17 +0000 UTC" deadline="2028-04-26 14:25:32.916975906 +0000 UTC"
Jul 17 13:45:18.774618 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.774302    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15576h40m14.142677609s"
Jul 17 13:45:18.774618 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.774386    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:18.774618 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:18.774509    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:18.784326 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.784294    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4" event={"ID":"ed26acdd-2775-46d9-b179-17def1c1d2f9","Type":"ContainerStarted","Data":"cff3a358507ef3615ef130bd3628e43917ffc61fffeab422163fcf229d6bbb97"}
Jul 17 13:45:18.787436 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.787363    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dztc6" event={"ID":"21b1cb43-5f8c-4bc6-9452-b6f39aa68e71","Type":"ContainerStarted","Data":"07bb9ec8dad4efe8727c4788833b41cac71021d6157eda54108254d2da23da87"}
Jul 17 13:45:18.794804 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.794776    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mpv6f" event={"ID":"5fd17b04-3478-47e5-bd54-66e8e7ee1cef","Type":"ContainerStarted","Data":"80c7a6531efdfdece33430d5ab3d6befb7452af63f00a7832147a7f7f51bf67a"}
Jul 17 13:45:18.798418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.798388    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-99rv5" event={"ID":"e1ae684d-4434-45b3-8075-749da32e5be8","Type":"ContainerStarted","Data":"663bb1df3446faabc730f7e38fbfe1c5cd7eea321c4f6c286b78e50b86b16e58"}
Jul 17 13:45:18.803176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.803147    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal" event={"ID":"ad89fec431916698e96755d7734948e7","Type":"ContainerStarted","Data":"9c6806c40eb8c343b25d8cecee31e863a4e0bf08e0e4183b86c1e86448ec8be0"}
Jul 17 13:45:18.813232 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.813209    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg" event={"ID":"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6","Type":"ContainerStarted","Data":"1ed9f1fbf5d85a10f11bf10ab4d00c71ba75000258c4659bf6e6ab215ce4fefd"}
Jul 17 13:45:18.817311 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.817254    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfkch" event={"ID":"dea22213-7df3-4fa7-a7c8-052bc5cbabbf","Type":"ContainerStarted","Data":"07ff5c0ccb023bb82a3026ea2879c22d4c08babaa6d1734b6c64d94507fed847"}
Jul 17 13:45:18.820059 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.820032    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xtx5f" event={"ID":"b0825032-81b7-4c8c-8494-cab86102a66c","Type":"ContainerStarted","Data":"7034f59071d16a56f37536769f98b15cfc222e8d843089ea68e5a213f4c66a83"}
Jul 17 13:45:18.822763 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.822687    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r7x4d" event={"ID":"b21695c3-88ac-45c2-bcce-e1a6bb654650","Type":"ContainerStarted","Data":"f7022abe27c26064be025b24b73aae96073f9f5fbb3ffcb7c55ea2074f46751b"}
Jul 17 13:45:18.829727 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.829035    2561 generic.go:356] "Generic (PLEG): container finished" podID="98840faf737d2cdde86a6e7fcaa940d5" containerID="329c223457304a59174b1d739ece73f66cac0281c0c87e06c2b7c1bcbd1ec935" exitCode=0
Jul 17 13:45:18.829727 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.829143    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal" event={"ID":"98840faf737d2cdde86a6e7fcaa940d5","Type":"ContainerDied","Data":"329c223457304a59174b1d739ece73f66cac0281c0c87e06c2b7c1bcbd1ec935"}
Jul 17 13:45:18.835341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.835317    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z77wd" event={"ID":"f222c7e6-2c4d-402a-879c-4a0ca6ae2475","Type":"ContainerStarted","Data":"208147fd9a86e947e9f802fb5bd770da9ec1acf2bcdb1546a3ec09aaef6a9602"}
Jul 17 13:45:18.861884 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:18.861819    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-163.ec2.internal" podStartSLOduration=2.861801876 podStartE2EDuration="2.861801876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:18.841201739 +0000 UTC m=+4.547648579" watchObservedRunningTime="2026-07-17 13:45:18.861801876 +0000 UTC m=+4.568248716"
Jul 17 13:45:19.765762 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:19.765721    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:19.765957 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:19.765860    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:19.863170 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:19.863136    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal" event={"ID":"98840faf737d2cdde86a6e7fcaa940d5","Type":"ContainerStarted","Data":"183a87b4cb7166275259ea01f13c885c8586573f905cf3d41e1fbe8d954430c5"}
Jul 17 13:45:20.267751 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:20.267712    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:20.267937 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:20.267890    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:20.267999 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:20.267957    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs podName:c5ffda97-3c84-479c-b0c2-d1c62ddc90d7 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:24.267936539 +0000 UTC m=+9.974383359 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs") pod "network-metrics-daemon-7dbv7" (UID: "c5ffda97-3c84-479c-b0c2-d1c62ddc90d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:20.368744 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:20.368699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:20.368924 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:20.368893    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:20.368987 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:20.368958    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret podName:f83860e4-2ac5-4187-aa2f-45fcc803faf5 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:24.368938628 +0000 UTC m=+10.075385451 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret") pod "global-pull-secret-syncer-gsr4m" (UID: "f83860e4-2ac5-4187-aa2f-45fcc803faf5") : object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:20.469510 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:20.469444    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sf7\" (UniqueName: \"kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7\") pod \"network-check-target-8gtz9\" (UID: \"f3ba7951-5c0a-4d39-8c81-a325d1abf091\") " pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:20.469699 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:20.469657    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 13:45:20.469699 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:20.469682    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 13:45:20.469699 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:20.469696    2561 projected.go:196] Error preparing data for projected volume kube-api-access-q2sf7 for pod openshift-network-diagnostics/network-check-target-8gtz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:20.469876 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:20.469759    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7 podName:f3ba7951-5c0a-4d39-8c81-a325d1abf091 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:24.469739557 +0000 UTC m=+10.176186376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2sf7" (UniqueName: "kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7") pod "network-check-target-8gtz9" (UID: "f3ba7951-5c0a-4d39-8c81-a325d1abf091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:20.768511 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:20.768482    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:20.768690 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:20.768658    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:20.769107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:20.769052    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:20.769198 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:20.769164    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:21.765781 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:21.765748    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:21.766276 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:21.765866    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:22.765806 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:22.765768    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:22.766283 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:22.765904    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:22.766341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:22.766309    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:22.766395 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:22.766380    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:23.765977 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:23.765472    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:23.765977 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:23.765622    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:24.299738 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:24.299702    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:24.299889 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:24.299842    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:24.299943 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:24.299904    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs podName:c5ffda97-3c84-479c-b0c2-d1c62ddc90d7 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:32.299884868 +0000 UTC m=+18.006331702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs") pod "network-metrics-daemon-7dbv7" (UID: "c5ffda97-3c84-479c-b0c2-d1c62ddc90d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:24.401033 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:24.400571    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:24.401033 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:24.400687    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:24.401033 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:24.400736    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret podName:f83860e4-2ac5-4187-aa2f-45fcc803faf5 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:32.400718242 +0000 UTC m=+18.107165053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret") pod "global-pull-secret-syncer-gsr4m" (UID: "f83860e4-2ac5-4187-aa2f-45fcc803faf5") : object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:24.501565 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:24.501476    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sf7\" (UniqueName: \"kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7\") pod \"network-check-target-8gtz9\" (UID: \"f3ba7951-5c0a-4d39-8c81-a325d1abf091\") " pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:24.501735 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:24.501662    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 13:45:24.501735 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:24.501688    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 13:45:24.501735 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:24.501701    2561 projected.go:196] Error preparing data for projected volume kube-api-access-q2sf7 for pod openshift-network-diagnostics/network-check-target-8gtz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:24.501896 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:24.501786    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7 podName:f3ba7951-5c0a-4d39-8c81-a325d1abf091 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:32.501767552 +0000 UTC m=+18.208214368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2sf7" (UniqueName: "kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7") pod "network-check-target-8gtz9" (UID: "f3ba7951-5c0a-4d39-8c81-a325d1abf091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:24.766963 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:24.766596    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:24.766963 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:24.766682    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:24.766963 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:24.766933    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:24.767488 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:24.767000    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:25.766242 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:25.766213    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:25.766434 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:25.766333    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:26.766096 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:26.766051    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:26.766548 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:26.766062    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:26.766548 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:26.766245    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:26.766548 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:26.766358    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:27.766065 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:27.766030    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:27.766258 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:27.766151    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:28.765622 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:28.765593    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:28.765789 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:28.765601    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:28.765789 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:28.765690    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:28.765789 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:28.765781    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:29.765567 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:29.765543    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:29.765985 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:29.765643    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:30.766131 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:30.766089    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:30.766604 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:30.766090    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:30.766604 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:30.766242    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:30.766604 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:30.766297    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:31.766023 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:31.765972    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:31.766243 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:31.766129    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:32.358245 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:32.358213    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:32.358491 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:32.358341    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:32.358491 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:32.358407    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs podName:c5ffda97-3c84-479c-b0c2-d1c62ddc90d7 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:48.358386669 +0000 UTC m=+34.064833482 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs") pod "network-metrics-daemon-7dbv7" (UID: "c5ffda97-3c84-479c-b0c2-d1c62ddc90d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:32.458828 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:32.458794    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:32.458992 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:32.458954    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:32.459082 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:32.459040    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret podName:f83860e4-2ac5-4187-aa2f-45fcc803faf5 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:48.459021389 +0000 UTC m=+34.165468216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret") pod "global-pull-secret-syncer-gsr4m" (UID: "f83860e4-2ac5-4187-aa2f-45fcc803faf5") : object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:32.559379 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:32.559351    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sf7\" (UniqueName: \"kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7\") pod \"network-check-target-8gtz9\" (UID: \"f3ba7951-5c0a-4d39-8c81-a325d1abf091\") " pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:32.559549 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:32.559511    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 13:45:32.559549 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:32.559533    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 13:45:32.559549 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:32.559544    2561 projected.go:196] Error preparing data for projected volume kube-api-access-q2sf7 for pod openshift-network-diagnostics/network-check-target-8gtz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:32.559697 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:32.559619    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7 podName:f3ba7951-5c0a-4d39-8c81-a325d1abf091 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:48.559585576 +0000 UTC m=+34.266032389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2sf7" (UniqueName: "kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7") pod "network-check-target-8gtz9" (UID: "f3ba7951-5c0a-4d39-8c81-a325d1abf091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:32.765768 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:32.765727    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:32.765932 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:32.765836    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:32.766066 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:32.766036    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:32.766199 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:32.766163    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:33.765434 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:33.765405    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:33.765882 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:33.765537    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:34.765997 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:34.765963    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:34.766432 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:34.766113    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:34.766432 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:34.766151    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:34.766432 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:34.766223    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:35.765372 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:35.765346    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:35.765546 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:35.765458    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:36.765752 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:36.765689    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:36.766324 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:36.765829    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:36.766324 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:36.765889    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:36.766324 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:36.766038    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:37.766358 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.766198    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:37.766912 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:37.766432    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:37.913294 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.913227    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mpv6f" event={"ID":"5fd17b04-3478-47e5-bd54-66e8e7ee1cef","Type":"ContainerStarted","Data":"a600c1e2e95562a023f33c91c678a3237f23bb814da5cb7697f8e9190ce27274"}
Jul 17 13:45:37.914885 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.914856    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg" event={"ID":"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6","Type":"ContainerStarted","Data":"2bf29ae4bcc23da17aa8068aa62ed3213d9625d0ab39cbe4d0bfef2821ad11e3"}
Jul 17 13:45:37.916778 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.916679    2561 generic.go:356] "Generic (PLEG): container finished" podID="dea22213-7df3-4fa7-a7c8-052bc5cbabbf" containerID="37ec016027035dadc22bed2e4335e0b30f7e9c72eca89a729e0ce9edd86d4008" exitCode=0
Jul 17 13:45:37.916778 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.916763    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfkch" event={"ID":"dea22213-7df3-4fa7-a7c8-052bc5cbabbf","Type":"ContainerDied","Data":"37ec016027035dadc22bed2e4335e0b30f7e9c72eca89a729e0ce9edd86d4008"}
Jul 17 13:45:37.918826 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.918803    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xtx5f" event={"ID":"b0825032-81b7-4c8c-8494-cab86102a66c","Type":"ContainerStarted","Data":"0135fca5c454c8a88a9fe3ac2e677e9686dc0c802eb7a0acf9845b59e50b0e85"}
Jul 17 13:45:37.920658 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.920567    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r7x4d" event={"ID":"b21695c3-88ac-45c2-bcce-e1a6bb654650","Type":"ContainerStarted","Data":"e4314e242466960dbbd3b465f7c3f30ade4ee552cd2cd685e4227a790597e7e5"}
Jul 17 13:45:37.923553 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.923526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z77wd" event={"ID":"f222c7e6-2c4d-402a-879c-4a0ca6ae2475","Type":"ContainerStarted","Data":"f17182cb272d57cdfdf3c6a7ec0fae88d5bde69c0d37a8131d9ee398d82b07fe"}
Jul 17 13:45:37.927703 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.927679    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4" event={"ID":"ed26acdd-2775-46d9-b179-17def1c1d2f9","Type":"ContainerStarted","Data":"832881a53f6b20d0fd1a71418ff40284aed6a7033b03d7f56e8717f5a8f70878"}
Jul 17 13:45:37.927798 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.927758    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4" event={"ID":"ed26acdd-2775-46d9-b179-17def1c1d2f9","Type":"ContainerStarted","Data":"c7cbdc90aac0ab6978a5f78120f34ab2404d0f47f47755b583af842e5a512c94"}
Jul 17 13:45:37.927859 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.927808    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4" event={"ID":"ed26acdd-2775-46d9-b179-17def1c1d2f9","Type":"ContainerStarted","Data":"008455a2432ccc37266fe898b3dec5cf336f0c80c9e4da827e3440af7a7d47d3"}
Jul 17 13:45:37.927859 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.927855    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4" event={"ID":"ed26acdd-2775-46d9-b179-17def1c1d2f9","Type":"ContainerStarted","Data":"2040a07beeb5ed42fdd675205407006703a6a86249c24718fbfa78e64b0c7bce"}
Jul 17 13:45:37.927948 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.927901    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4" event={"ID":"ed26acdd-2775-46d9-b179-17def1c1d2f9","Type":"ContainerStarted","Data":"8bde411622710109896c270a64f98dc9e39e49d2594faf3169fc69e116a3764f"}
Jul 17 13:45:37.927986 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.927947    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4" event={"ID":"ed26acdd-2775-46d9-b179-17def1c1d2f9","Type":"ContainerStarted","Data":"58e8a56e53877853e59c5e4b2e0397cb3ce9987011f34994fa956ef8641a61e5"}
Jul 17 13:45:37.929239 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.929216    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dztc6" event={"ID":"21b1cb43-5f8c-4bc6-9452-b6f39aa68e71","Type":"ContainerStarted","Data":"9c836b49559d0feb57660d4d5282313eddecdf5ff81e27a81fcabca2f98a8aaf"}
Jul 17 13:45:37.933987 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.933953    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-163.ec2.internal" podStartSLOduration=21.9339408 podStartE2EDuration="21.9339408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:19.883623506 +0000 UTC m=+5.590070346" watchObservedRunningTime="2026-07-17 13:45:37.9339408 +0000 UTC m=+23.640387634"
Jul 17 13:45:37.934162 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.934135    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mpv6f" podStartSLOduration=3.7308928249999997 podStartE2EDuration="22.934127732s" totalImagesPullingTime="19.203234907s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:37.933298131 +0000 UTC m=+23.639744967" watchObservedRunningTime="2026-07-17 13:45:37.934127732 +0000 UTC m=+23.640574566"
Jul 17 13:45:37.951890 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.951850    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z77wd" podStartSLOduration=3.793868635 podStartE2EDuration="22.951837657s" totalImagesPullingTime="19.157969022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:37.951256825 +0000 UTC m=+23.657703659" watchObservedRunningTime="2026-07-17 13:45:37.951837657 +0000 UTC m=+23.658284491"
Jul 17 13:45:37.998871 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.998848    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:37.999564 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:37.999544    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:38.002565 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.002526    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xtx5f" podStartSLOduration=3.81632615 podStartE2EDuration="23.002513715s" totalImagesPullingTime="19.186187565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:37.971383625 +0000 UTC m=+23.677830459" watchObservedRunningTime="2026-07-17 13:45:38.002513715 +0000 UTC m=+23.708960594"
Jul 17 13:45:38.002758 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.002736    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r7x4d" podStartSLOduration=3.779514723 podStartE2EDuration="23.002731486s" totalImagesPullingTime="19.223216763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:38.000357705 +0000 UTC m=+23.706804559" watchObservedRunningTime="2026-07-17 13:45:38.002731486 +0000 UTC m=+23.709178319"
Jul 17 13:45:38.018780 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.018741    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dztc6" podStartSLOduration=8.091272741 podStartE2EDuration="23.018728475s" totalImagesPullingTime="14.927455734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:38.018495973 +0000 UTC m=+23.724942807" watchObservedRunningTime="2026-07-17 13:45:38.018728475 +0000 UTC m=+23.725175308"
Jul 17 13:45:38.598961 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.598925    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 13:45:38.696210 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.696083    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T13:45:38.598956176Z","UUID":"a836b255-0c12-4bfb-ab20-f5fd6ff0d1b2","Handler":null,"Name":"","Endpoint":""}
Jul 17 13:45:38.697677 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.697659    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 13:45:38.697777 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.697701    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 13:45:38.766232 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.766198    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:38.766410 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.766204    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:38.766410 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:38.766312    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:38.766850 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:38.766400    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:38.933259 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.933225    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-99rv5" event={"ID":"e1ae684d-4434-45b3-8075-749da32e5be8","Type":"ContainerStarted","Data":"48ff0c4436ac14134713e831bc7263c461975a7ccf0652d8fc186e4ce8b698ff"}
Jul 17 13:45:38.935771 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.935741    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg" event={"ID":"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6","Type":"ContainerStarted","Data":"2839fcbfa3a52932747e0915c2fe1ac389bc8c494759f9fe5349316c300ca5dc"}
Jul 17 13:45:38.936109 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.936095    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:38.936668 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.936647    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dztc6"
Jul 17 13:45:38.954937 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:38.954854    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-99rv5" podStartSLOduration=4.7932696 podStartE2EDuration="23.954836554s" totalImagesPullingTime="19.161566954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:38.954030354 +0000 UTC m=+24.660477190" watchObservedRunningTime="2026-07-17 13:45:38.954836554 +0000 UTC m=+24.661283390"
Jul 17 13:45:39.766130 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:39.766100    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:39.766349 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:39.766220    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:39.940123 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:39.940095    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg" event={"ID":"0f0af0eb-6443-4fb4-8137-e0179c0ab7c6","Type":"ContainerStarted","Data":"a21baaed27b69e3753ea40ad6c2229dbcd7d167e5341d22670ad379be2ca7694"}
Jul 17 13:45:39.983024 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:39.982959    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b6hlg" podStartSLOduration=3.538679902 podStartE2EDuration="24.982937162s" totalImagesPullingTime="21.44425726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:39.982807851 +0000 UTC m=+25.689254686" watchObservedRunningTime="2026-07-17 13:45:39.982937162 +0000 UTC m=+25.689384183"
Jul 17 13:45:40.765944 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:40.765908    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:40.766129 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:40.766065    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:40.766192 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:40.766123    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:40.766274 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:40.766249    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:40.946421 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:40.946379    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4" event={"ID":"ed26acdd-2775-46d9-b179-17def1c1d2f9","Type":"ContainerStarted","Data":"7ec2c1e05c5222931b0825995fa0ff386c89e824e56a2722cb071ba775264499"}
Jul 17 13:45:41.765993 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:41.765963    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:41.766192 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:41.766095    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:42.765574 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:42.765414    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:42.766149 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:42.765462    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:42.766149 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:42.765662    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:42.766149 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:42.765744    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:42.954262 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:42.954228    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4" event={"ID":"ed26acdd-2775-46d9-b179-17def1c1d2f9","Type":"ContainerStarted","Data":"cfdea39924dee3356f3fe913116f91313b911eb59bd6dce691b31fec295c5f42"}
Jul 17 13:45:42.954512 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:42.954493    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:42.957328 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:42.957305    2561 generic.go:356] "Generic (PLEG): container finished" podID="dea22213-7df3-4fa7-a7c8-052bc5cbabbf" containerID="1c7cb1cd63d6427ddfdb541ece1173f681fd3879c3aedcba2175b8798a03fb93" exitCode=0
Jul 17 13:45:42.957446 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:42.957339    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfkch" event={"ID":"dea22213-7df3-4fa7-a7c8-052bc5cbabbf","Type":"ContainerDied","Data":"1c7cb1cd63d6427ddfdb541ece1173f681fd3879c3aedcba2175b8798a03fb93"}
Jul 17 13:45:42.970304 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:42.970285    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:42.989628 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:42.989589    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4" podStartSLOduration=8.758507613999999 podStartE2EDuration="27.989577627s" totalImagesPullingTime="19.231070013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:42.989232163 +0000 UTC m=+28.695678999" watchObservedRunningTime="2026-07-17 13:45:42.989577627 +0000 UTC m=+28.696024464"
Jul 17 13:45:43.060368 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:43.060345    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:43.766101 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:43.765845    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:43.766518 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:43.766208    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:43.959684 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:43.959646    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:43.975025 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:43.974986    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:45:44.437996 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:44.437966    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gsr4m"]
Jul 17 13:45:44.438248 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:44.438091    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:44.438248 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:44.438198    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:44.438714 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:44.438677    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8gtz9"]
Jul 17 13:45:44.438854 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:44.438799    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:44.438917 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:44.438882    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:44.439304 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:44.439286    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7dbv7"]
Jul 17 13:45:44.439406 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:44.439392    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:44.439517 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:44.439500    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:44.963209 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:44.963174    2561 generic.go:356] "Generic (PLEG): container finished" podID="dea22213-7df3-4fa7-a7c8-052bc5cbabbf" containerID="6a1adf4395a5fd9c33e0a4c5505ca623dccc044569db472d5acad3a30896daa6" exitCode=0
Jul 17 13:45:44.963633 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:44.963250    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfkch" event={"ID":"dea22213-7df3-4fa7-a7c8-052bc5cbabbf","Type":"ContainerDied","Data":"6a1adf4395a5fd9c33e0a4c5505ca623dccc044569db472d5acad3a30896daa6"}
Jul 17 13:45:45.766258 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:45.766225    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:45.766422 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:45.766290    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:45.766422 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:45.766288    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:45.766422 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:45.766370    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:45.766574 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:45.766441    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:45.766574 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:45.766495    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:46.971540 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:46.971434    2561 generic.go:356] "Generic (PLEG): container finished" podID="dea22213-7df3-4fa7-a7c8-052bc5cbabbf" containerID="dd762a0d284d3fc5d8968e79ed83e90d9aea0a7f41a746c8a60c2c8329e496a7" exitCode=0
Jul 17 13:45:46.971893 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:46.971539    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfkch" event={"ID":"dea22213-7df3-4fa7-a7c8-052bc5cbabbf","Type":"ContainerDied","Data":"dd762a0d284d3fc5d8968e79ed83e90d9aea0a7f41a746c8a60c2c8329e496a7"}
Jul 17 13:45:47.766149 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:47.765909    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:47.766344 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:47.765910    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:47.766344 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:47.766222    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:47.766456 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:47.766348    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:47.766456 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:47.765958    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:47.766563 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:47.766486    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:48.377696 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:48.377662    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:48.378088 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:48.377793    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:48.378088 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:48.377849    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs podName:c5ffda97-3c84-479c-b0c2-d1c62ddc90d7 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:20.377835195 +0000 UTC m=+66.084282007 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs") pod "network-metrics-daemon-7dbv7" (UID: "c5ffda97-3c84-479c-b0c2-d1c62ddc90d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 13:45:48.478054 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:48.477997    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:48.478203 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:48.478150    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:48.478269 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:48.478212    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret podName:f83860e4-2ac5-4187-aa2f-45fcc803faf5 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:20.478198281 +0000 UTC m=+66.184645097 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret") pod "global-pull-secret-syncer-gsr4m" (UID: "f83860e4-2ac5-4187-aa2f-45fcc803faf5") : object "kube-system"/"original-pull-secret" not registered
Jul 17 13:45:48.579315 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:48.579285    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sf7\" (UniqueName: \"kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7\") pod \"network-check-target-8gtz9\" (UID: \"f3ba7951-5c0a-4d39-8c81-a325d1abf091\") " pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:48.579477 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:48.579459    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 13:45:48.579477 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:48.579477    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 13:45:48.579618 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:48.579489    2561 projected.go:196] Error preparing data for projected volume kube-api-access-q2sf7 for pod openshift-network-diagnostics/network-check-target-8gtz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:48.579618 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:48.579539    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7 podName:f3ba7951-5c0a-4d39-8c81-a325d1abf091 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:20.579526036 +0000 UTC m=+66.285972847 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2sf7" (UniqueName: "kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7") pod "network-check-target-8gtz9" (UID: "f3ba7951-5c0a-4d39-8c81-a325d1abf091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 13:45:49.765738 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:49.765659    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:49.766176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:49.765778    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:49.766176 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:49.765785    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsr4m" podUID="f83860e4-2ac5-4187-aa2f-45fcc803faf5"
Jul 17 13:45:49.766176 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:49.765892    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:45:49.766176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:49.765788    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:49.766176 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:49.765975    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8gtz9" podUID="f3ba7951-5c0a-4d39-8c81-a325d1abf091"
Jul 17 13:45:51.195046 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.195000    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-163.ec2.internal" event="NodeReady"
Jul 17 13:45:51.195308 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.195150    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 13:45:51.329956 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.329899    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5qnvt"]
Jul 17 13:45:51.331608 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.331594    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5qnvt"
Jul 17 13:45:51.342294 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.342273    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8r6d5"]
Jul 17 13:45:51.344103 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.344087    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.346351 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.346324    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 13:45:51.346451 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.346357    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 13:45:51.346451 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.346380    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v8874\""
Jul 17 13:45:51.346451 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.346424    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 13:45:51.354568 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.354551    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 13:45:51.356653 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.356639    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 13:45:51.357077 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.357064    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gl8vw\""
Jul 17 13:45:51.377713 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.377691    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5qnvt"]
Jul 17 13:45:51.397322 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.397296    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/decf2204-ccbc-4915-81f8-009c80ac9398-cert\") pod \"ingress-canary-5qnvt\" (UID: \"decf2204-ccbc-4915-81f8-009c80ac9398\") " pod="openshift-ingress-canary/ingress-canary-5qnvt"
Jul 17 13:45:51.397422 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.397340    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j57cp\" (UniqueName: \"kubernetes.io/projected/decf2204-ccbc-4915-81f8-009c80ac9398-kube-api-access-j57cp\") pod \"ingress-canary-5qnvt\" (UID: \"decf2204-ccbc-4915-81f8-009c80ac9398\") " pod="openshift-ingress-canary/ingress-canary-5qnvt"
Jul 17 13:45:51.402720 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.402700    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8r6d5"]
Jul 17 13:45:51.498296 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.498270    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/474b65de-97a7-487f-80f2-0d4c5b3d5df1-config-volume\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.498419 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.498299    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-825gs\" (UniqueName: \"kubernetes.io/projected/474b65de-97a7-487f-80f2-0d4c5b3d5df1-kube-api-access-825gs\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.498419 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.498323    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/decf2204-ccbc-4915-81f8-009c80ac9398-cert\") pod \"ingress-canary-5qnvt\" (UID: \"decf2204-ccbc-4915-81f8-009c80ac9398\") " pod="openshift-ingress-canary/ingress-canary-5qnvt"
Jul 17 13:45:51.498419 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.498397    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j57cp\" (UniqueName: \"kubernetes.io/projected/decf2204-ccbc-4915-81f8-009c80ac9398-kube-api-access-j57cp\") pod \"ingress-canary-5qnvt\" (UID: \"decf2204-ccbc-4915-81f8-009c80ac9398\") " pod="openshift-ingress-canary/ingress-canary-5qnvt"
Jul 17 13:45:51.498564 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.498422    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.498564 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.498448    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/474b65de-97a7-487f-80f2-0d4c5b3d5df1-tmp-dir\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.502593 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.502569    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/decf2204-ccbc-4915-81f8-009c80ac9398-cert\") pod \"ingress-canary-5qnvt\" (UID: \"decf2204-ccbc-4915-81f8-009c80ac9398\") " pod="openshift-ingress-canary/ingress-canary-5qnvt"
Jul 17 13:45:51.511116 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.511099    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j57cp\" (UniqueName: \"kubernetes.io/projected/decf2204-ccbc-4915-81f8-009c80ac9398-kube-api-access-j57cp\") pod \"ingress-canary-5qnvt\" (UID: \"decf2204-ccbc-4915-81f8-009c80ac9398\") " pod="openshift-ingress-canary/ingress-canary-5qnvt"
Jul 17 13:45:51.599457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.599413    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/474b65de-97a7-487f-80f2-0d4c5b3d5df1-config-volume\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.599457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.599437    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-825gs\" (UniqueName: \"kubernetes.io/projected/474b65de-97a7-487f-80f2-0d4c5b3d5df1-kube-api-access-825gs\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.599580 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.599462    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.599580 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.599480    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/474b65de-97a7-487f-80f2-0d4c5b3d5df1-tmp-dir\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.599580 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:51.599570    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 13:45:51.599692 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:51.599631    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls podName:474b65de-97a7-487f-80f2-0d4c5b3d5df1 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:52.099612749 +0000 UTC m=+37.806059586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls") pod "dns-default-8r6d5" (UID: "474b65de-97a7-487f-80f2-0d4c5b3d5df1") : secret "dns-default-metrics-tls" not found
Jul 17 13:45:51.599769 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.599755    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/474b65de-97a7-487f-80f2-0d4c5b3d5df1-tmp-dir\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.599926 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.599909    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/474b65de-97a7-487f-80f2-0d4c5b3d5df1-config-volume\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.613320 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.613290    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-825gs\" (UniqueName: \"kubernetes.io/projected/474b65de-97a7-487f-80f2-0d4c5b3d5df1-kube-api-access-825gs\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:51.640117 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.640098    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5qnvt"
Jul 17 13:45:51.765522 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.765498    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:45:51.765649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.765503    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:45:51.765649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.765626    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:45:51.768494 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.768477    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 13:45:51.769258 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.769239    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xnsb\""
Jul 17 13:45:51.769333 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.769308    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tpk25\""
Jul 17 13:45:51.769399 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.769385    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 13:45:51.770039 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.769998    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 13:45:51.770632 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.770620    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 13:45:51.785212 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.785192    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5qnvt"]
Jul 17 13:45:51.789211 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:45:51.789187    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddecf2204_ccbc_4915_81f8_009c80ac9398.slice/crio-81493a2bbe77b11f86a6d6d3fc3d6fdc1b89859afaa8cea5d37725116fe82060 WatchSource:0}: Error finding container 81493a2bbe77b11f86a6d6d3fc3d6fdc1b89859afaa8cea5d37725116fe82060: Status 404 returned error can't find the container with id 81493a2bbe77b11f86a6d6d3fc3d6fdc1b89859afaa8cea5d37725116fe82060
Jul 17 13:45:51.984835 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.984809    2561 generic.go:356] "Generic (PLEG): container finished" podID="dea22213-7df3-4fa7-a7c8-052bc5cbabbf" containerID="5b7cecb8123e1f8e093fcc18b5aa184776a4bd6ede2288eea0322050fce9245f" exitCode=0
Jul 17 13:45:51.984971 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.984905    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfkch" event={"ID":"dea22213-7df3-4fa7-a7c8-052bc5cbabbf","Type":"ContainerDied","Data":"5b7cecb8123e1f8e093fcc18b5aa184776a4bd6ede2288eea0322050fce9245f"}
Jul 17 13:45:51.986052 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:51.986033    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5qnvt" event={"ID":"decf2204-ccbc-4915-81f8-009c80ac9398","Type":"ContainerStarted","Data":"81493a2bbe77b11f86a6d6d3fc3d6fdc1b89859afaa8cea5d37725116fe82060"}
Jul 17 13:45:52.103024 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:52.102861    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:52.103134 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:52.103066    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 13:45:52.103134 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:52.103128    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls podName:474b65de-97a7-487f-80f2-0d4c5b3d5df1 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:53.103111555 +0000 UTC m=+38.809558367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls") pod "dns-default-8r6d5" (UID: "474b65de-97a7-487f-80f2-0d4c5b3d5df1") : secret "dns-default-metrics-tls" not found
Jul 17 13:45:52.991064 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:52.991028    2561 generic.go:356] "Generic (PLEG): container finished" podID="dea22213-7df3-4fa7-a7c8-052bc5cbabbf" containerID="38ee0306aeadc4dbdd85a23f7a8e198d076f586178d2a19feadcc8b12fd429c8" exitCode=0
Jul 17 13:45:52.991668 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:52.991079    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfkch" event={"ID":"dea22213-7df3-4fa7-a7c8-052bc5cbabbf","Type":"ContainerDied","Data":"38ee0306aeadc4dbdd85a23f7a8e198d076f586178d2a19feadcc8b12fd429c8"}
Jul 17 13:45:53.110504 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:53.110470    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:53.110634 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:53.110609    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 13:45:53.110711 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:53.110663    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls podName:474b65de-97a7-487f-80f2-0d4c5b3d5df1 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:55.11064471 +0000 UTC m=+40.817091538 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls") pod "dns-default-8r6d5" (UID: "474b65de-97a7-487f-80f2-0d4c5b3d5df1") : secret "dns-default-metrics-tls" not found
Jul 17 13:45:53.995949 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:53.995920    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfkch" event={"ID":"dea22213-7df3-4fa7-a7c8-052bc5cbabbf","Type":"ContainerStarted","Data":"84996a59d4b8f36a285028b7459e23c6863a2ed6ea1fba9cfcb1d694f1563367"}
Jul 17 13:45:54.021945 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:54.021896    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dfkch" podStartSLOduration=9.064579604 podStartE2EDuration="39.021879242s" totalImagesPullingTime="29.957299638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:54.021305153 +0000 UTC m=+39.727751997" watchObservedRunningTime="2026-07-17 13:45:54.021879242 +0000 UTC m=+39.728326077"
Jul 17 13:45:54.999805 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:54.999770    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5qnvt" event={"ID":"decf2204-ccbc-4915-81f8-009c80ac9398","Type":"ContainerStarted","Data":"c89451dc16d8d7b8a7de64627d7cb2c30e1c4bb9d7e7f5af70937a108a16375f"}
Jul 17 13:45:55.017827 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:55.017783    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5qnvt" podStartSLOduration=1.569518741 podStartE2EDuration="4.017769501s" totalImagesPullingTime="2.44825076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:45:55.017544539 +0000 UTC m=+40.723991376" watchObservedRunningTime="2026-07-17 13:45:55.017769501 +0000 UTC m=+40.724216335"
Jul 17 13:45:55.127094 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:55.127058    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:55.127291 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:55.127271    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 13:45:55.127361 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:55.127351    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls podName:474b65de-97a7-487f-80f2-0d4c5b3d5df1 nodeName:}" failed. No retries permitted until 2026-07-17 13:45:59.127330579 +0000 UTC m=+44.833777392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls") pod "dns-default-8r6d5" (UID: "474b65de-97a7-487f-80f2-0d4c5b3d5df1") : secret "dns-default-metrics-tls" not found
Jul 17 13:45:59.153205 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:45:59.153173    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:45:59.153545 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:59.153314    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 13:45:59.153545 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:45:59.153369    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls podName:474b65de-97a7-487f-80f2-0d4c5b3d5df1 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:07.153354818 +0000 UTC m=+52.859801634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls") pod "dns-default-8r6d5" (UID: "474b65de-97a7-487f-80f2-0d4c5b3d5df1") : secret "dns-default-metrics-tls" not found
Jul 17 13:46:07.208996 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:07.208965    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:46:07.209534 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:46:07.209095    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 13:46:07.209534 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:46:07.209154    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls podName:474b65de-97a7-487f-80f2-0d4c5b3d5df1 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:23.209140496 +0000 UTC m=+68.915587307 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls") pod "dns-default-8r6d5" (UID: "474b65de-97a7-487f-80f2-0d4c5b3d5df1") : secret "dns-default-metrics-tls" not found
Jul 17 13:46:15.983364 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:15.983336    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s56n4"
Jul 17 13:46:20.393974 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.393938    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:46:20.396876 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.396859    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 13:46:20.404970 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:46:20.404950    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 13:46:20.405044 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:46:20.405032    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs podName:c5ffda97-3c84-479c-b0c2-d1c62ddc90d7 nodeName:}" failed. No retries permitted until 2026-07-17 13:47:24.404997037 +0000 UTC m=+130.111443865 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs") pod "network-metrics-daemon-7dbv7" (UID: "c5ffda97-3c84-479c-b0c2-d1c62ddc90d7") : secret "metrics-daemon-secret" not found
Jul 17 13:46:20.494870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.494844    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:46:20.497562 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.497548    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 13:46:20.509100 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.509078    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f83860e4-2ac5-4187-aa2f-45fcc803faf5-original-pull-secret\") pod \"global-pull-secret-syncer-gsr4m\" (UID: \"f83860e4-2ac5-4187-aa2f-45fcc803faf5\") " pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:46:20.586306 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.586287    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsr4m"
Jul 17 13:46:20.595816 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.595795    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sf7\" (UniqueName: \"kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7\") pod \"network-check-target-8gtz9\" (UID: \"f3ba7951-5c0a-4d39-8c81-a325d1abf091\") " pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:46:20.598751 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.598726    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 13:46:20.608390 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.608365    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 13:46:20.618908 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.618889    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2sf7\" (UniqueName: \"kubernetes.io/projected/f3ba7951-5c0a-4d39-8c81-a325d1abf091-kube-api-access-q2sf7\") pod \"network-check-target-8gtz9\" (UID: \"f3ba7951-5c0a-4d39-8c81-a325d1abf091\") " pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:46:20.722715 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.722688    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gsr4m"]
Jul 17 13:46:20.725731 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:46:20.725699    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83860e4_2ac5_4187_aa2f_45fcc803faf5.slice/crio-468b8cc389f4753a38aea0619d03f11e2956ae8dd0fae7d79b2dfe1cc8154d10 WatchSource:0}: Error finding container 468b8cc389f4753a38aea0619d03f11e2956ae8dd0fae7d79b2dfe1cc8154d10: Status 404 returned error can't find the container with id 468b8cc389f4753a38aea0619d03f11e2956ae8dd0fae7d79b2dfe1cc8154d10
Jul 17 13:46:20.884523 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.884498    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tpk25\""
Jul 17 13:46:20.892815 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:20.892796    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:46:21.011507 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:21.011479    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8gtz9"]
Jul 17 13:46:21.014328 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:46:21.014293    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ba7951_5c0a_4d39_8c81_a325d1abf091.slice/crio-1100005f2e8f80fa0dc75a74445831a44ef5ead4fdcfdaf3f75c1dbc1dcd38d9 WatchSource:0}: Error finding container 1100005f2e8f80fa0dc75a74445831a44ef5ead4fdcfdaf3f75c1dbc1dcd38d9: Status 404 returned error can't find the container with id 1100005f2e8f80fa0dc75a74445831a44ef5ead4fdcfdaf3f75c1dbc1dcd38d9
Jul 17 13:46:21.057596 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:21.057558    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8gtz9" event={"ID":"f3ba7951-5c0a-4d39-8c81-a325d1abf091","Type":"ContainerStarted","Data":"1100005f2e8f80fa0dc75a74445831a44ef5ead4fdcfdaf3f75c1dbc1dcd38d9"}
Jul 17 13:46:21.059069 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:21.059035    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gsr4m" event={"ID":"f83860e4-2ac5-4187-aa2f-45fcc803faf5","Type":"ContainerStarted","Data":"468b8cc389f4753a38aea0619d03f11e2956ae8dd0fae7d79b2dfe1cc8154d10"}
Jul 17 13:46:23.214582 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:23.214538    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:46:23.215082 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:46:23.214768    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 13:46:23.215082 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:46:23.214842    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls podName:474b65de-97a7-487f-80f2-0d4c5b3d5df1 nodeName:}" failed. No retries permitted until 2026-07-17 13:46:55.214814827 +0000 UTC m=+100.921261640 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls") pod "dns-default-8r6d5" (UID: "474b65de-97a7-487f-80f2-0d4c5b3d5df1") : secret "dns-default-metrics-tls" not found
Jul 17 13:46:26.075045 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:26.074940    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gsr4m" event={"ID":"f83860e4-2ac5-4187-aa2f-45fcc803faf5","Type":"ContainerStarted","Data":"cc92bfc3951ebff171eaa004680e8a17271095be8f4bcee019489f10b74a3a2b"}
Jul 17 13:46:26.076344 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:26.076313    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8gtz9" event={"ID":"f3ba7951-5c0a-4d39-8c81-a325d1abf091","Type":"ContainerStarted","Data":"6cd702861dab485fd45e2e12a99c9206e3b58e1cab3c8f983617d7bda29260dc"}
Jul 17 13:46:26.076476 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:26.076428    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:46:26.093890 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:26.093840    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gsr4m" podStartSLOduration=66.116506113 podStartE2EDuration="1m11.093825296s" totalImagesPullingTime="4.977319183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:46:26.093396098 +0000 UTC m=+71.799843156" watchObservedRunningTime="2026-07-17 13:46:26.093825296 +0000 UTC m=+71.800272133"
Jul 17 13:46:26.109202 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:26.109162    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8gtz9" podStartSLOduration=66.420581571 podStartE2EDuration="1m11.109150836s" totalImagesPullingTime="4.688569265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:46:26.108843543 +0000 UTC m=+71.815290376" watchObservedRunningTime="2026-07-17 13:46:26.109150836 +0000 UTC m=+71.815597670"
Jul 17 13:46:55.226048 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:55.225985    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:46:55.226609 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:46:55.226152    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 13:46:55.226609 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:46:55.226217    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls podName:474b65de-97a7-487f-80f2-0d4c5b3d5df1 nodeName:}" failed. No retries permitted until 2026-07-17 13:47:59.226201727 +0000 UTC m=+164.932648539 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls") pod "dns-default-8r6d5" (UID: "474b65de-97a7-487f-80f2-0d4c5b3d5df1") : secret "dns-default-metrics-tls" not found
Jul 17 13:46:57.081113 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:46:57.081083    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8gtz9"
Jul 17 13:47:24.405242 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:47:24.405201    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:47:24.405720 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:47:24.405330    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 13:47:24.405720 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:47:24.405384    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs podName:c5ffda97-3c84-479c-b0c2-d1c62ddc90d7 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:26.405370911 +0000 UTC m=+252.111817724 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs") pod "network-metrics-daemon-7dbv7" (UID: "c5ffda97-3c84-479c-b0c2-d1c62ddc90d7") : secret "metrics-daemon-secret" not found
Jul 17 13:47:54.351935 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:47:54.351890    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8r6d5" podUID="474b65de-97a7-487f-80f2-0d4c5b3d5df1"
Jul 17 13:47:54.775632 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:47:54.775600    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:47:55.282188 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:47:55.282159    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:47:59.227615 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:47:59.227585    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:47:59.228056 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:47:59.227699    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 13:47:59.228056 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:47:59.227749    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls podName:474b65de-97a7-487f-80f2-0d4c5b3d5df1 nodeName:}" failed. No retries permitted until 2026-07-17 13:50:01.227735227 +0000 UTC m=+286.934182042 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls") pod "dns-default-8r6d5" (UID: "474b65de-97a7-487f-80f2-0d4c5b3d5df1") : secret "dns-default-metrics-tls" not found
Jul 17 13:48:07.765752 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:48:07.765666    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:49:26.487348 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:26.487314    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:49:26.487920 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:26.487420    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 13:49:26.487920 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:26.487475    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs podName:c5ffda97-3c84-479c-b0c2-d1c62ddc90d7 nodeName:}" failed. No retries permitted until 2026-07-17 13:51:28.487461962 +0000 UTC m=+374.193908774 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs") pod "network-metrics-daemon-7dbv7" (UID: "c5ffda97-3c84-479c-b0c2-d1c62ddc90d7") : secret "metrics-daemon-secret" not found
Jul 17 13:49:36.825028 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.824924    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw"]
Jul 17 13:49:36.826714 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.826696    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw"
Jul 17 13:49:36.830138 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.830120    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qdqll\""
Jul 17 13:49:36.846526 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.846505    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-kp8xd"]
Jul 17 13:49:36.848106 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.848086    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw"]
Jul 17 13:49:36.848190 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.848162    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.852877 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.852855    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-serving-cert\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.852985 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.852886    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.852985 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.852902    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-snapshots\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.853088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.852983    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-service-ca-bundle\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.853088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.853042    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-tmp\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.853088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.853060    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfm55\" (UniqueName: \"kubernetes.io/projected/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-kube-api-access-rfm55\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.853088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.853079    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6qwm\" (UniqueName: \"kubernetes.io/projected/4ea51dab-267f-4aab-b0d0-ab0a7e6b215a-kube-api-access-h6qwm\") pod \"network-check-source-7bc5b6bf85-5t9qw\" (UID: \"4ea51dab-267f-4aab-b0d0-ab0a7e6b215a\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw"
Jul 17 13:49:36.856555 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.856537    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 17 13:49:36.856647 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.856574    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 13:49:36.858359 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.858341    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 13:49:36.858786 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.858774    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 17 13:49:36.862157 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.862141    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2fktf\""
Jul 17 13:49:36.865063 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.865049    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 17 13:49:36.869192 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.869173    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-kp8xd"]
Jul 17 13:49:36.932870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.932818    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"]
Jul 17 13:49:36.934819 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.934802    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-558ff794f6-txt9j"]
Jul 17 13:49:36.934938 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.934925    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:36.936586 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.936567    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:36.948998 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.948976    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 17 13:49:36.949103 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.949035    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 17 13:49:36.949103 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.949052    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 17 13:49:36.949103 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.949068    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 17 13:49:36.949103 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.949077    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 17 13:49:36.949241 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.949118    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 13:49:36.949241 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.949196    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 17 13:49:36.949241 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.949202    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 17 13:49:36.949241 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.949207    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 13:49:36.949423 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.949265    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 17 13:49:36.949423 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.949274    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-txjhc\""
Jul 17 13:49:36.950049 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.950033    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tclk9\""
Jul 17 13:49:36.953757 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.953739    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-service-ca-bundle\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.953845 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.953765    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:36.953845 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.953784    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-tmp\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.953845 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.953799    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfm55\" (UniqueName: \"kubernetes.io/projected/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-kube-api-access-rfm55\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.953845 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.953821    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzfmq\" (UniqueName: \"kubernetes.io/projected/6e15a328-f305-4234-af3e-5c3779bbb526-kube-api-access-pzfmq\") pod \"service-ca-operator-5bcbc94c78-w9wg8\" (UID: \"6e15a328-f305-4234-af3e-5c3779bbb526\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:36.953845 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.953844    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqljk\" (UniqueName: \"kubernetes.io/projected/332b1551-2e12-4d8b-8b34-55aeba43b9f3-kube-api-access-vqljk\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:36.954137 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.953899    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6qwm\" (UniqueName: \"kubernetes.io/projected/4ea51dab-267f-4aab-b0d0-ab0a7e6b215a-kube-api-access-h6qwm\") pod \"network-check-source-7bc5b6bf85-5t9qw\" (UID: \"4ea51dab-267f-4aab-b0d0-ab0a7e6b215a\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw"
Jul 17 13:49:36.954190 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.954149    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:36.954190 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.954184    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-stats-auth\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:36.954290 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.954231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-serving-cert\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.954290 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.954237    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-tmp\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.954290 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.954269    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-default-certificate\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:36.954447 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.954310    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.954447 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.954343    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-snapshots\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.954447 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.954400    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e15a328-f305-4234-af3e-5c3779bbb526-serving-cert\") pod \"service-ca-operator-5bcbc94c78-w9wg8\" (UID: \"6e15a328-f305-4234-af3e-5c3779bbb526\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:36.954447 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.954423    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e15a328-f305-4234-af3e-5c3779bbb526-config\") pod \"service-ca-operator-5bcbc94c78-w9wg8\" (UID: \"6e15a328-f305-4234-af3e-5c3779bbb526\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:36.954447 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.954443    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-service-ca-bundle\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.955125 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.955101    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-snapshots\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.955632 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.955588    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.955993 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.955976    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"]
Jul 17 13:49:36.956831 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.956802    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-serving-cert\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.957662 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.957645    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-558ff794f6-txt9j"]
Jul 17 13:49:36.976415 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.976392    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfm55\" (UniqueName: \"kubernetes.io/projected/a04aa92c-47f8-4dd2-bc0f-cb8604321dc8-kube-api-access-rfm55\") pod \"insights-operator-555d9d7885-kp8xd\" (UID: \"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8\") " pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:36.982717 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:36.982698    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6qwm\" (UniqueName: \"kubernetes.io/projected/4ea51dab-267f-4aab-b0d0-ab0a7e6b215a-kube-api-access-h6qwm\") pod \"network-check-source-7bc5b6bf85-5t9qw\" (UID: \"4ea51dab-267f-4aab-b0d0-ab0a7e6b215a\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw"
Jul 17 13:49:37.054678 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.054649    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e15a328-f305-4234-af3e-5c3779bbb526-serving-cert\") pod \"service-ca-operator-5bcbc94c78-w9wg8\" (UID: \"6e15a328-f305-4234-af3e-5c3779bbb526\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:37.054678 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.054676    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e15a328-f305-4234-af3e-5c3779bbb526-config\") pod \"service-ca-operator-5bcbc94c78-w9wg8\" (UID: \"6e15a328-f305-4234-af3e-5c3779bbb526\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:37.054851 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.054707    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:37.054851 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.054726    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzfmq\" (UniqueName: \"kubernetes.io/projected/6e15a328-f305-4234-af3e-5c3779bbb526-kube-api-access-pzfmq\") pod \"service-ca-operator-5bcbc94c78-w9wg8\" (UID: \"6e15a328-f305-4234-af3e-5c3779bbb526\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:37.054851 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.054744    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqljk\" (UniqueName: \"kubernetes.io/projected/332b1551-2e12-4d8b-8b34-55aeba43b9f3-kube-api-access-vqljk\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:37.054851 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:37.054802    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 13:49:37.055073 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.054859    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:37.055073 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.054891    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-stats-auth\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:37.055073 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:37.054910    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs podName:332b1551-2e12-4d8b-8b34-55aeba43b9f3 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:37.554889245 +0000 UTC m=+263.261336070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs") pod "router-default-558ff794f6-txt9j" (UID: "332b1551-2e12-4d8b-8b34-55aeba43b9f3") : secret "router-metrics-certs-default" not found
Jul 17 13:49:37.055073 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.054977    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-default-certificate\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:37.055073 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:37.054998    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle podName:332b1551-2e12-4d8b-8b34-55aeba43b9f3 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:37.554982997 +0000 UTC m=+263.261429810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle") pod "router-default-558ff794f6-txt9j" (UID: "332b1551-2e12-4d8b-8b34-55aeba43b9f3") : configmap references non-existent config key: service-ca.crt
Jul 17 13:49:37.055381 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.055298    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e15a328-f305-4234-af3e-5c3779bbb526-config\") pod \"service-ca-operator-5bcbc94c78-w9wg8\" (UID: \"6e15a328-f305-4234-af3e-5c3779bbb526\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:37.056952 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.056936    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e15a328-f305-4234-af3e-5c3779bbb526-serving-cert\") pod \"service-ca-operator-5bcbc94c78-w9wg8\" (UID: \"6e15a328-f305-4234-af3e-5c3779bbb526\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:37.057189 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.057174    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-default-certificate\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:37.057369 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.057351    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-stats-auth\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:37.067740 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.067722    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzfmq\" (UniqueName: \"kubernetes.io/projected/6e15a328-f305-4234-af3e-5c3779bbb526-kube-api-access-pzfmq\") pod \"service-ca-operator-5bcbc94c78-w9wg8\" (UID: \"6e15a328-f305-4234-af3e-5c3779bbb526\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:37.069457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.069441    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqljk\" (UniqueName: \"kubernetes.io/projected/332b1551-2e12-4d8b-8b34-55aeba43b9f3-kube-api-access-vqljk\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:37.135117 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.135076    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw"
Jul 17 13:49:37.155572 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.155553    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-kp8xd"
Jul 17 13:49:37.244430 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.244401    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"
Jul 17 13:49:37.267588 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.267487    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw"]
Jul 17 13:49:37.270993 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:49:37.270967    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea51dab_267f_4aab_b0d0_ab0a7e6b215a.slice/crio-648ed4fbade77803fc7ccc6df03984d80979f331cb26876a55de656059042209 WatchSource:0}: Error finding container 648ed4fbade77803fc7ccc6df03984d80979f331cb26876a55de656059042209: Status 404 returned error can't find the container with id 648ed4fbade77803fc7ccc6df03984d80979f331cb26876a55de656059042209
Jul 17 13:49:37.289763 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.289735    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-kp8xd"]
Jul 17 13:49:37.298239 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:49:37.298204    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04aa92c_47f8_4dd2_bc0f_cb8604321dc8.slice/crio-b4376614276d1647e3598bcc190f55ca30aff635a292bc314a31418121d33385 WatchSource:0}: Error finding container b4376614276d1647e3598bcc190f55ca30aff635a292bc314a31418121d33385: Status 404 returned error can't find the container with id b4376614276d1647e3598bcc190f55ca30aff635a292bc314a31418121d33385
Jul 17 13:49:37.374095 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.374069    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8"]
Jul 17 13:49:37.377102 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:49:37.377074    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e15a328_f305_4234_af3e_5c3779bbb526.slice/crio-39f806028330166959952ef2678d0159768b74786829101d8ec124902414697d WatchSource:0}: Error finding container 39f806028330166959952ef2678d0159768b74786829101d8ec124902414697d: Status 404 returned error can't find the container with id 39f806028330166959952ef2678d0159768b74786829101d8ec124902414697d
Jul 17 13:49:37.515574 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.515549    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-kp8xd" event={"ID":"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8","Type":"ContainerStarted","Data":"b4376614276d1647e3598bcc190f55ca30aff635a292bc314a31418121d33385"}
Jul 17 13:49:37.517024 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.516970    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw" event={"ID":"4ea51dab-267f-4aab-b0d0-ab0a7e6b215a","Type":"ContainerStarted","Data":"769804573e63893037fe9da577701d3c1418040391674f435c31603b6955a60c"}
Jul 17 13:49:37.517137 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.517043    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw" event={"ID":"4ea51dab-267f-4aab-b0d0-ab0a7e6b215a","Type":"ContainerStarted","Data":"648ed4fbade77803fc7ccc6df03984d80979f331cb26876a55de656059042209"}
Jul 17 13:49:37.518105 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.518079    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8" event={"ID":"6e15a328-f305-4234-af3e-5c3779bbb526","Type":"ContainerStarted","Data":"39f806028330166959952ef2678d0159768b74786829101d8ec124902414697d"}
Jul 17 13:49:37.534334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.534296    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5t9qw" podStartSLOduration=1.534282352 podStartE2EDuration="1.534282352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:49:37.533226082 +0000 UTC m=+263.239672915" watchObservedRunningTime="2026-07-17 13:49:37.534282352 +0000 UTC m=+263.240729185"
Jul 17 13:49:37.557962 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.557934    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:37.558097 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:37.557992    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:37.558097 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:37.558090    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 13:49:37.558210 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:37.558141    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle podName:332b1551-2e12-4d8b-8b34-55aeba43b9f3 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:38.558123047 +0000 UTC m=+264.264569879 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle") pod "router-default-558ff794f6-txt9j" (UID: "332b1551-2e12-4d8b-8b34-55aeba43b9f3") : configmap references non-existent config key: service-ca.crt
Jul 17 13:49:37.558210 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:37.558162    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs podName:332b1551-2e12-4d8b-8b34-55aeba43b9f3 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:38.55815245 +0000 UTC m=+264.264599275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs") pod "router-default-558ff794f6-txt9j" (UID: "332b1551-2e12-4d8b-8b34-55aeba43b9f3") : secret "router-metrics-certs-default" not found
Jul 17 13:49:38.567669 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:38.567633    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:38.568099 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:38.567699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:38.568099 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:38.567778    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 13:49:38.568099 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:38.567827    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle podName:332b1551-2e12-4d8b-8b34-55aeba43b9f3 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:40.567809477 +0000 UTC m=+266.274256289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle") pod "router-default-558ff794f6-txt9j" (UID: "332b1551-2e12-4d8b-8b34-55aeba43b9f3") : configmap references non-existent config key: service-ca.crt
Jul 17 13:49:38.568099 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:38.567843    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs podName:332b1551-2e12-4d8b-8b34-55aeba43b9f3 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:40.56783664 +0000 UTC m=+266.274283453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs") pod "router-default-558ff794f6-txt9j" (UID: "332b1551-2e12-4d8b-8b34-55aeba43b9f3") : secret "router-metrics-certs-default" not found
Jul 17 13:49:40.529475 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:40.529443    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8" event={"ID":"6e15a328-f305-4234-af3e-5c3779bbb526","Type":"ContainerStarted","Data":"62250cfc8ade207db014db26821f9928c3192df92300571bbab96f74f6182e19"}
Jul 17 13:49:40.530880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:40.530859    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-kp8xd" event={"ID":"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8","Type":"ContainerStarted","Data":"b88cf9862291a1d825f88bbda4345ae3ba735a0b2974e0c5c8cf1aaf57183204"}
Jul 17 13:49:40.557299 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:40.557255    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8" podStartSLOduration=2.038659058 podStartE2EDuration="4.557235359s" totalImagesPullingTime="2.518576301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:49:40.556398029 +0000 UTC m=+266.262844862" watchObservedRunningTime="2026-07-17 13:49:40.557235359 +0000 UTC m=+266.263682194"
Jul 17 13:49:40.584020 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:40.583387    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:40.584020 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:40.583473    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:40.584020 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:40.583850    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 13:49:40.584020 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:40.583906    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs podName:332b1551-2e12-4d8b-8b34-55aeba43b9f3 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:44.583888395 +0000 UTC m=+270.290335209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs") pod "router-default-558ff794f6-txt9j" (UID: "332b1551-2e12-4d8b-8b34-55aeba43b9f3") : secret "router-metrics-certs-default" not found
Jul 17 13:49:40.584020 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:40.583975    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle podName:332b1551-2e12-4d8b-8b34-55aeba43b9f3 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:44.583963949 +0000 UTC m=+270.290410764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle") pod "router-default-558ff794f6-txt9j" (UID: "332b1551-2e12-4d8b-8b34-55aeba43b9f3") : configmap references non-existent config key: service-ca.crt
Jul 17 13:49:40.588243 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:40.588198    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-kp8xd" podStartSLOduration=1.98991884 podStartE2EDuration="4.588184973s" totalImagesPullingTime="2.598266133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:49:40.58771994 +0000 UTC m=+266.294166775" watchObservedRunningTime="2026-07-17 13:49:40.588184973 +0000 UTC m=+266.294631807"
Jul 17 13:49:41.408432 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.408403    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kmb5l"]
Jul 17 13:49:41.413278 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.413243    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.419742 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.419722    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 13:49:41.420000 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.419986    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kldnd\""
Jul 17 13:49:41.420690 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.420677    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 13:49:41.490420 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.490391    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcffd48f-2a19-40a0-a545-ae16a7f763bc-data-volume\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.490572 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.490442    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dcffd48f-2a19-40a0-a545-ae16a7f763bc-crio-socket\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.490572 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.490459    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.490572 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.490508    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x827r\" (UniqueName: \"kubernetes.io/projected/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-api-access-x827r\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.490572 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.490532    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.511583 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.511561    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kmb5l"]
Jul 17 13:49:41.591180 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.591151    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dcffd48f-2a19-40a0-a545-ae16a7f763bc-crio-socket\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.591602 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.591188    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.591602 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:41.591287    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 13:49:41.591602 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.591289    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dcffd48f-2a19-40a0-a545-ae16a7f763bc-crio-socket\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.591602 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:41.591355    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm podName:dcffd48f-2a19-40a0-a545-ae16a7f763bc nodeName:}" failed. No retries permitted until 2026-07-17 13:49:42.091341001 +0000 UTC m=+267.797787815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmb5l" (UID: "dcffd48f-2a19-40a0-a545-ae16a7f763bc") : configmap "kube-rbac-proxy" not found
Jul 17 13:49:41.591602 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.591423    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x827r\" (UniqueName: \"kubernetes.io/projected/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-api-access-x827r\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.591602 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.591471    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.591602 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.591504    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcffd48f-2a19-40a0-a545-ae16a7f763bc-data-volume\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.591942 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:41.591686    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 13:49:41.591942 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:41.591750    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls podName:dcffd48f-2a19-40a0-a545-ae16a7f763bc nodeName:}" failed. No retries permitted until 2026-07-17 13:49:42.091737836 +0000 UTC m=+267.798184651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmb5l" (UID: "dcffd48f-2a19-40a0-a545-ae16a7f763bc") : secret "insights-runtime-extractor-tls" not found
Jul 17 13:49:41.591942 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.591860    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcffd48f-2a19-40a0-a545-ae16a7f763bc-data-volume\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:41.609975 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:41.609944    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x827r\" (UniqueName: \"kubernetes.io/projected/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-api-access-x827r\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:42.095486 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:42.095442    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:42.095667 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:42.095520    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:42.095667 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:42.095561    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 13:49:42.095667 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:42.095631    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 13:49:42.095667 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:42.095660    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm podName:dcffd48f-2a19-40a0-a545-ae16a7f763bc nodeName:}" failed. No retries permitted until 2026-07-17 13:49:43.095620797 +0000 UTC m=+268.802067612 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmb5l" (UID: "dcffd48f-2a19-40a0-a545-ae16a7f763bc") : configmap "kube-rbac-proxy" not found
Jul 17 13:49:42.095805 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:42.095675    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls podName:dcffd48f-2a19-40a0-a545-ae16a7f763bc nodeName:}" failed. No retries permitted until 2026-07-17 13:49:43.095668937 +0000 UTC m=+268.802115749 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmb5l" (UID: "dcffd48f-2a19-40a0-a545-ae16a7f763bc") : secret "insights-runtime-extractor-tls" not found
Jul 17 13:49:43.101399 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:43.101365    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:43.101835 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:43.101454    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:43.101835 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:43.101535    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 13:49:43.101835 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:43.101619    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls podName:dcffd48f-2a19-40a0-a545-ae16a7f763bc nodeName:}" failed. No retries permitted until 2026-07-17 13:49:45.101597745 +0000 UTC m=+270.808044574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmb5l" (UID: "dcffd48f-2a19-40a0-a545-ae16a7f763bc") : secret "insights-runtime-extractor-tls" not found
Jul 17 13:49:43.101835 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:43.101539    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 13:49:43.101835 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:43.101673    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm podName:dcffd48f-2a19-40a0-a545-ae16a7f763bc nodeName:}" failed. No retries permitted until 2026-07-17 13:49:45.101657028 +0000 UTC m=+270.808103845 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmb5l" (UID: "dcffd48f-2a19-40a0-a545-ae16a7f763bc") : configmap "kube-rbac-proxy" not found
Jul 17 13:49:43.549278 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:43.549248    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z77wd_f222c7e6-2c4d-402a-879c-4a0ca6ae2475/dns-node-resolver/0.log"
Jul 17 13:49:44.087446 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.087419    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-sd2fz"]
Jul 17 13:49:44.090474 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.090459    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.093115 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.093098    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 17 13:49:44.094085 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.094061    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 17 13:49:44.094194 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.094101    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-njn5t\""
Jul 17 13:49:44.094194 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.094130    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 17 13:49:44.094194 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.094142    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 17 13:49:44.098277 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.098257    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-sd2fz"]
Jul 17 13:49:44.108276 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.108257    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a2af0e1e-3dae-421d-a197-ef84b10eb95c-signing-key\") pod \"service-ca-58598695f7-sd2fz\" (UID: \"a2af0e1e-3dae-421d-a197-ef84b10eb95c\") " pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.108532 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.108330    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzv7k\" (UniqueName: \"kubernetes.io/projected/a2af0e1e-3dae-421d-a197-ef84b10eb95c-kube-api-access-fzv7k\") pod \"service-ca-58598695f7-sd2fz\" (UID: \"a2af0e1e-3dae-421d-a197-ef84b10eb95c\") " pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.108532 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.108384    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a2af0e1e-3dae-421d-a197-ef84b10eb95c-signing-cabundle\") pod \"service-ca-58598695f7-sd2fz\" (UID: \"a2af0e1e-3dae-421d-a197-ef84b10eb95c\") " pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.209158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.209135    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzv7k\" (UniqueName: \"kubernetes.io/projected/a2af0e1e-3dae-421d-a197-ef84b10eb95c-kube-api-access-fzv7k\") pod \"service-ca-58598695f7-sd2fz\" (UID: \"a2af0e1e-3dae-421d-a197-ef84b10eb95c\") " pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.209265 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.209163    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a2af0e1e-3dae-421d-a197-ef84b10eb95c-signing-cabundle\") pod \"service-ca-58598695f7-sd2fz\" (UID: \"a2af0e1e-3dae-421d-a197-ef84b10eb95c\") " pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.209265 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.209199    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a2af0e1e-3dae-421d-a197-ef84b10eb95c-signing-key\") pod \"service-ca-58598695f7-sd2fz\" (UID: \"a2af0e1e-3dae-421d-a197-ef84b10eb95c\") " pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.209721 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.209700    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a2af0e1e-3dae-421d-a197-ef84b10eb95c-signing-cabundle\") pod \"service-ca-58598695f7-sd2fz\" (UID: \"a2af0e1e-3dae-421d-a197-ef84b10eb95c\") " pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.211564 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.211539    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a2af0e1e-3dae-421d-a197-ef84b10eb95c-signing-key\") pod \"service-ca-58598695f7-sd2fz\" (UID: \"a2af0e1e-3dae-421d-a197-ef84b10eb95c\") " pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.217249 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.217231    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzv7k\" (UniqueName: \"kubernetes.io/projected/a2af0e1e-3dae-421d-a197-ef84b10eb95c-kube-api-access-fzv7k\") pod \"service-ca-58598695f7-sd2fz\" (UID: \"a2af0e1e-3dae-421d-a197-ef84b10eb95c\") " pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.399637 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.399566    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-sd2fz"
Jul 17 13:49:44.520711 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.520683    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-sd2fz"]
Jul 17 13:49:44.523517 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:49:44.523495    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2af0e1e_3dae_421d_a197_ef84b10eb95c.slice/crio-e0e69372fcc67c559c2dd1a18c2a9c441196631af319ebddc962151fe0ef1ffe WatchSource:0}: Error finding container e0e69372fcc67c559c2dd1a18c2a9c441196631af319ebddc962151fe0ef1ffe: Status 404 returned error can't find the container with id e0e69372fcc67c559c2dd1a18c2a9c441196631af319ebddc962151fe0ef1ffe
Jul 17 13:49:44.542929 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.542905    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-sd2fz" event={"ID":"a2af0e1e-3dae-421d-a197-ef84b10eb95c","Type":"ContainerStarted","Data":"e0e69372fcc67c559c2dd1a18c2a9c441196631af319ebddc962151fe0ef1ffe"}
Jul 17 13:49:44.549338 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.549321    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xtx5f_b0825032-81b7-4c8c-8494-cab86102a66c/node-ca/0.log"
Jul 17 13:49:44.613272 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.613247    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:44.613383 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.613287    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:44.613430 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:44.613398    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 13:49:44.613465 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:44.613401    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle podName:332b1551-2e12-4d8b-8b34-55aeba43b9f3 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:52.613386391 +0000 UTC m=+278.319833203 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle") pod "router-default-558ff794f6-txt9j" (UID: "332b1551-2e12-4d8b-8b34-55aeba43b9f3") : configmap references non-existent config key: service-ca.crt
Jul 17 13:49:44.613465 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:44.613455    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs podName:332b1551-2e12-4d8b-8b34-55aeba43b9f3 nodeName:}" failed. No retries permitted until 2026-07-17 13:49:52.613437523 +0000 UTC m=+278.319884335 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs") pod "router-default-558ff794f6-txt9j" (UID: "332b1551-2e12-4d8b-8b34-55aeba43b9f3") : secret "router-metrics-certs-default" not found
Jul 17 13:49:44.950546 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:44.950517    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5qnvt_decf2204-ccbc-4915-81f8-009c80ac9398/serve-healthcheck-canary/0.log"
Jul 17 13:49:45.116271 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:45.116230    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:45.116271 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:45.116281    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:45.116646 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:45.116390    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 13:49:45.116646 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:45.116398    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 13:49:45.116646 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:45.116452    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm podName:dcffd48f-2a19-40a0-a545-ae16a7f763bc nodeName:}" failed. No retries permitted until 2026-07-17 13:49:49.116437733 +0000 UTC m=+274.822884547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmb5l" (UID: "dcffd48f-2a19-40a0-a545-ae16a7f763bc") : configmap "kube-rbac-proxy" not found
Jul 17 13:49:45.116646 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:45.116465    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls podName:dcffd48f-2a19-40a0-a545-ae16a7f763bc nodeName:}" failed. No retries permitted until 2026-07-17 13:49:49.11645955 +0000 UTC m=+274.822906361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmb5l" (UID: "dcffd48f-2a19-40a0-a545-ae16a7f763bc") : secret "insights-runtime-extractor-tls" not found
Jul 17 13:49:45.547115 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:45.547085    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-sd2fz" event={"ID":"a2af0e1e-3dae-421d-a197-ef84b10eb95c","Type":"ContainerStarted","Data":"bf1f6812a7fa0622f0c4c1e1554f6111cf60aceb136cedcb9831866d62734b24"}
Jul 17 13:49:45.564903 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:45.564849    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-sd2fz" podStartSLOduration=1.5648314920000002 podStartE2EDuration="1.564831492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:49:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:49:45.563487125 +0000 UTC m=+271.269933980" watchObservedRunningTime="2026-07-17 13:49:45.564831492 +0000 UTC m=+271.271278327"
Jul 17 13:49:49.147358 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:49.147318    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:49.147766 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:49.147363    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:49.147766 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:49.147461    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 13:49:49.147766 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:49.147469    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 13:49:49.147766 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:49.147526    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls podName:dcffd48f-2a19-40a0-a545-ae16a7f763bc nodeName:}" failed. No retries permitted until 2026-07-17 13:49:57.147511489 +0000 UTC m=+282.853958301 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmb5l" (UID: "dcffd48f-2a19-40a0-a545-ae16a7f763bc") : secret "insights-runtime-extractor-tls" not found
Jul 17 13:49:49.147766 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:49.147540    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm podName:dcffd48f-2a19-40a0-a545-ae16a7f763bc nodeName:}" failed. No retries permitted until 2026-07-17 13:49:57.147532883 +0000 UTC m=+282.853979694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmb5l" (UID: "dcffd48f-2a19-40a0-a545-ae16a7f763bc") : configmap "kube-rbac-proxy" not found
Jul 17 13:49:52.671368 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:52.671326    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:52.671728 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:52.671378    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:52.672023 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:52.671985    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/332b1551-2e12-4d8b-8b34-55aeba43b9f3-service-ca-bundle\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:52.673741 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:52.673710    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332b1551-2e12-4d8b-8b34-55aeba43b9f3-metrics-certs\") pod \"router-default-558ff794f6-txt9j\" (UID: \"332b1551-2e12-4d8b-8b34-55aeba43b9f3\") " pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:52.848327 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:52.848294    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:52.983889 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:52.983860    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-558ff794f6-txt9j"]
Jul 17 13:49:52.986759 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:49:52.986733    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod332b1551_2e12_4d8b_8b34_55aeba43b9f3.slice/crio-7a25311a52314a21267e805c2c2926c73104c37b0031c88c093d9af93f7a1c39 WatchSource:0}: Error finding container 7a25311a52314a21267e805c2c2926c73104c37b0031c88c093d9af93f7a1c39: Status 404 returned error can't find the container with id 7a25311a52314a21267e805c2c2926c73104c37b0031c88c093d9af93f7a1c39
Jul 17 13:49:53.575192 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:53.575156    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-558ff794f6-txt9j" event={"ID":"332b1551-2e12-4d8b-8b34-55aeba43b9f3","Type":"ContainerStarted","Data":"38b8f157c5545bc916822ef4e9dad07ac0b2d4b022697145ea896754af1fac37"}
Jul 17 13:49:53.575385 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:53.575210    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-558ff794f6-txt9j" event={"ID":"332b1551-2e12-4d8b-8b34-55aeba43b9f3","Type":"ContainerStarted","Data":"7a25311a52314a21267e805c2c2926c73104c37b0031c88c093d9af93f7a1c39"}
Jul 17 13:49:53.594639 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:53.594582    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-558ff794f6-txt9j" podStartSLOduration=17.594565452 podStartE2EDuration="17.594565452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:49:53.59439889 +0000 UTC m=+279.300845725" watchObservedRunningTime="2026-07-17 13:49:53.594565452 +0000 UTC m=+279.301012297"
Jul 17 13:49:53.849088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:53.848998    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:53.851373 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:53.851351    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:54.578775 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:54.578741    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:54.579848 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:54.579826    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-558ff794f6-txt9j"
Jul 17 13:49:57.204208 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:57.204169    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:57.204208 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:57.204210    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:57.204644 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:57.204628    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcffd48f-2a19-40a0-a545-ae16a7f763bc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:57.206498 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:57.206479    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcffd48f-2a19-40a0-a545-ae16a7f763bc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmb5l\" (UID: \"dcffd48f-2a19-40a0-a545-ae16a7f763bc\") " pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:57.322108 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:57.322074    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kmb5l"
Jul 17 13:49:57.455568 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:57.455496    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kmb5l"]
Jul 17 13:49:57.457840 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:49:57.457817    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcffd48f_2a19_40a0_a545_ae16a7f763bc.slice/crio-860e461491599ec0be5511c0332fdb83f67a821162651635648d47dee30aed30 WatchSource:0}: Error finding container 860e461491599ec0be5511c0332fdb83f67a821162651635648d47dee30aed30: Status 404 returned error can't find the container with id 860e461491599ec0be5511c0332fdb83f67a821162651635648d47dee30aed30
Jul 17 13:49:57.588396 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:57.588367    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmb5l" event={"ID":"dcffd48f-2a19-40a0-a545-ae16a7f763bc","Type":"ContainerStarted","Data":"860e461491599ec0be5511c0332fdb83f67a821162651635648d47dee30aed30"}
Jul 17 13:49:58.283048 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:49:58.282985    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8r6d5" podUID="474b65de-97a7-487f-80f2-0d4c5b3d5df1"
Jul 17 13:49:58.594079 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:49:58.593991    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:50:01.235182 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:01.235143    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:50:01.237626 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:01.237603    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/474b65de-97a7-487f-80f2-0d4c5b3d5df1-metrics-tls\") pod \"dns-default-8r6d5\" (UID: \"474b65de-97a7-487f-80f2-0d4c5b3d5df1\") " pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:50:01.297527 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:01.297498    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gl8vw\""
Jul 17 13:50:01.305065 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:01.305035    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:50:02.397050 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:02.397028    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8r6d5"]
Jul 17 13:50:02.400234 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:02.400207    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod474b65de_97a7_487f_80f2_0d4c5b3d5df1.slice/crio-ebea6282145a0ab7d69ae7373106610cfa778877d4a07ff452acfea36d8bdf65 WatchSource:0}: Error finding container ebea6282145a0ab7d69ae7373106610cfa778877d4a07ff452acfea36d8bdf65: Status 404 returned error can't find the container with id ebea6282145a0ab7d69ae7373106610cfa778877d4a07ff452acfea36d8bdf65
Jul 17 13:50:02.608864 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:02.608786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8r6d5" event={"ID":"474b65de-97a7-487f-80f2-0d4c5b3d5df1","Type":"ContainerStarted","Data":"ebea6282145a0ab7d69ae7373106610cfa778877d4a07ff452acfea36d8bdf65"}
Jul 17 13:50:02.610111 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:02.610080    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmb5l" event={"ID":"dcffd48f-2a19-40a0-a545-ae16a7f763bc","Type":"ContainerStarted","Data":"cb09be5300787bc9e9bada42f4c0f0dede9ae66a1f262767808819abd3b0800a"}
Jul 17 13:50:06.626021 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:06.625945    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8r6d5" event={"ID":"474b65de-97a7-487f-80f2-0d4c5b3d5df1","Type":"ContainerStarted","Data":"6b2d8647cd30a59bc2a906b65657762401f7e5992357bce09f14e96ab8721a3b"}
Jul 17 13:50:06.626484 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:06.626027    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8r6d5" event={"ID":"474b65de-97a7-487f-80f2-0d4c5b3d5df1","Type":"ContainerStarted","Data":"947e34101d620a6e47836e0c06b49b7a8d41a0514fbb565f0100e36c9f06f0b0"}
Jul 17 13:50:06.626484 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:06.626056    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:50:06.627833 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:06.627803    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmb5l" event={"ID":"dcffd48f-2a19-40a0-a545-ae16a7f763bc","Type":"ContainerStarted","Data":"6673812e0db360b7b664e776ad1f5798de319e4e7a2a68a27cbd5c2b9e94a52d"}
Jul 17 13:50:06.645657 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:06.645609    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8r6d5" podStartSLOduration=252.049618353 podStartE2EDuration="4m15.645591024s" totalImagesPullingTime="3.595972671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:06.644592005 +0000 UTC m=+292.351038843" watchObservedRunningTime="2026-07-17 13:50:06.645591024 +0000 UTC m=+292.352037860"
Jul 17 13:50:06.994830 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:06.994796    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d879b7597-vdcs5"]
Jul 17 13:50:06.997792 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:06.997768    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.006107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.006082    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 17 13:50:07.006247 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.006168    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cnxqs\""
Jul 17 13:50:07.006992 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.006971    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 17 13:50:07.007347 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.007323    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 17 13:50:07.011723 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.011703    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 17 13:50:07.013556 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.013534    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d879b7597-vdcs5"]
Jul 17 13:50:07.084811 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.084781    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c538f37-6861-475b-9681-4190bccfb2d9-registry-certificates\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.084984 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.084825    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c538f37-6861-475b-9681-4190bccfb2d9-bound-sa-token\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.084984 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.084850    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c538f37-6861-475b-9681-4190bccfb2d9-ca-trust-extracted\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.084984 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.084923    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c538f37-6861-475b-9681-4190bccfb2d9-trusted-ca\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.084984 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.084969    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c538f37-6861-475b-9681-4190bccfb2d9-image-registry-private-configuration\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.085214 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.085016    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c538f37-6861-475b-9681-4190bccfb2d9-installation-pull-secrets\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.085214 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.085048    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c538f37-6861-475b-9681-4190bccfb2d9-registry-tls\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.085214 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.085065    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9ttz\" (UniqueName: \"kubernetes.io/projected/1c538f37-6861-475b-9681-4190bccfb2d9-kube-api-access-j9ttz\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.185447 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.185409    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c538f37-6861-475b-9681-4190bccfb2d9-installation-pull-secrets\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.185604 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.185454    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c538f37-6861-475b-9681-4190bccfb2d9-registry-tls\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.185604 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.185478    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9ttz\" (UniqueName: \"kubernetes.io/projected/1c538f37-6861-475b-9681-4190bccfb2d9-kube-api-access-j9ttz\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.185604 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.185532    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c538f37-6861-475b-9681-4190bccfb2d9-registry-certificates\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.185604 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.185558    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c538f37-6861-475b-9681-4190bccfb2d9-bound-sa-token\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.185604 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.185579    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c538f37-6861-475b-9681-4190bccfb2d9-ca-trust-extracted\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.185858 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.185632    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c538f37-6861-475b-9681-4190bccfb2d9-trusted-ca\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.185858 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.185671    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c538f37-6861-475b-9681-4190bccfb2d9-image-registry-private-configuration\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.186793 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.186634    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c538f37-6861-475b-9681-4190bccfb2d9-ca-trust-extracted\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.186793 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.186714    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c538f37-6861-475b-9681-4190bccfb2d9-registry-certificates\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.187044 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.187000    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c538f37-6861-475b-9681-4190bccfb2d9-trusted-ca\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.188490 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.188470    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c538f37-6861-475b-9681-4190bccfb2d9-registry-tls\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.188740 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.188715    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c538f37-6861-475b-9681-4190bccfb2d9-installation-pull-secrets\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.189346 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.189313    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c538f37-6861-475b-9681-4190bccfb2d9-image-registry-private-configuration\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.194924 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.194901    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c538f37-6861-475b-9681-4190bccfb2d9-bound-sa-token\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.195532 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.195516    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9ttz\" (UniqueName: \"kubernetes.io/projected/1c538f37-6861-475b-9681-4190bccfb2d9-kube-api-access-j9ttz\") pod \"image-registry-6d879b7597-vdcs5\" (UID: \"1c538f37-6861-475b-9681-4190bccfb2d9\") " pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.309293 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.309223    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.472865 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.472835    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d879b7597-vdcs5"]
Jul 17 13:50:07.476852 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:07.476815    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c538f37_6861_475b_9681_4190bccfb2d9.slice/crio-a9cff71cc47b4d36a45b6f66e23148565f924e1a207fd462046ef82a4c7ef869 WatchSource:0}: Error finding container a9cff71cc47b4d36a45b6f66e23148565f924e1a207fd462046ef82a4c7ef869: Status 404 returned error can't find the container with id a9cff71cc47b4d36a45b6f66e23148565f924e1a207fd462046ef82a4c7ef869
Jul 17 13:50:07.633680 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.633583    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d879b7597-vdcs5" event={"ID":"1c538f37-6861-475b-9681-4190bccfb2d9","Type":"ContainerStarted","Data":"36b32b33280779ff96436e5db7740bce42a8b39a4c2b63a00c22fbbf4242eec7"}
Jul 17 13:50:07.633680 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.633661    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d879b7597-vdcs5" event={"ID":"1c538f37-6861-475b-9681-4190bccfb2d9","Type":"ContainerStarted","Data":"a9cff71cc47b4d36a45b6f66e23148565f924e1a207fd462046ef82a4c7ef869"}
Jul 17 13:50:07.634189 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.633968    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:07.656561 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:07.656500    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d879b7597-vdcs5" podStartSLOduration=1.656482212 podStartE2EDuration="1.656482212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:50:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:07.656131041 +0000 UTC m=+293.362577902" watchObservedRunningTime="2026-07-17 13:50:07.656482212 +0000 UTC m=+293.362929047"
Jul 17 13:50:09.642577 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:09.642548    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmb5l" event={"ID":"dcffd48f-2a19-40a0-a545-ae16a7f763bc","Type":"ContainerStarted","Data":"3605c6f9823ef6f45f73388dbf0906443fccb3cabb1d7880deaa68f1a30fca31"}
Jul 17 13:50:09.662185 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:09.662134    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kmb5l" podStartSLOduration=16.664450526 podStartE2EDuration="28.662118869s" totalImagesPullingTime="11.997668343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:49:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:09.661776376 +0000 UTC m=+295.368223220" watchObservedRunningTime="2026-07-17 13:50:09.662118869 +0000 UTC m=+295.368565712"
Jul 17 13:50:10.766894 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:10.766858    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7dbv7" podUID="c5ffda97-3c84-479c-b0c2-d1c62ddc90d7"
Jul 17 13:50:11.751207 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:11.751172    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"]
Jul 17 13:50:11.754270 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:11.754255    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"
Jul 17 13:50:11.757558 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:11.757534    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 17 13:50:11.757558 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:11.757545    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-g5kkg\""
Jul 17 13:50:11.763196 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:11.763176    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"]
Jul 17 13:50:11.819954 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:11.819936    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/637bfa7c-ee64-4429-8726-b0281760e30b-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-nkvs5\" (UID: \"637bfa7c-ee64-4429-8726-b0281760e30b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"
Jul 17 13:50:11.920623 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:11.920601    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/637bfa7c-ee64-4429-8726-b0281760e30b-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-nkvs5\" (UID: \"637bfa7c-ee64-4429-8726-b0281760e30b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"
Jul 17 13:50:11.920718 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:11.920705    2561 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 17 13:50:11.920762 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:11.920755    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/637bfa7c-ee64-4429-8726-b0281760e30b-tls-certificates podName:637bfa7c-ee64-4429-8726-b0281760e30b nodeName:}" failed. No retries permitted until 2026-07-17 13:50:12.42074129 +0000 UTC m=+298.127188102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/637bfa7c-ee64-4429-8726-b0281760e30b-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-nkvs5" (UID: "637bfa7c-ee64-4429-8726-b0281760e30b") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 17 13:50:12.423457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:12.423431    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/637bfa7c-ee64-4429-8726-b0281760e30b-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-nkvs5\" (UID: \"637bfa7c-ee64-4429-8726-b0281760e30b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"
Jul 17 13:50:12.425896 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:12.425870    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/637bfa7c-ee64-4429-8726-b0281760e30b-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-nkvs5\" (UID: \"637bfa7c-ee64-4429-8726-b0281760e30b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"
Jul 17 13:50:12.664133 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:12.664105    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"
Jul 17 13:50:12.788300 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:12.788272    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"]
Jul 17 13:50:12.791364 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:12.791337    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod637bfa7c_ee64_4429_8726_b0281760e30b.slice/crio-84ecfa400bcec65a5198b2fd063ac505aa24b5aace4267d23b783bf0e7b030f1 WatchSource:0}: Error finding container 84ecfa400bcec65a5198b2fd063ac505aa24b5aace4267d23b783bf0e7b030f1: Status 404 returned error can't find the container with id 84ecfa400bcec65a5198b2fd063ac505aa24b5aace4267d23b783bf0e7b030f1
Jul 17 13:50:13.655622 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:13.655584    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5" event={"ID":"637bfa7c-ee64-4429-8726-b0281760e30b","Type":"ContainerStarted","Data":"84ecfa400bcec65a5198b2fd063ac505aa24b5aace4267d23b783bf0e7b030f1"}
Jul 17 13:50:14.645998 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:14.645966    2561 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 13:50:14.659360 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:14.659329    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5" event={"ID":"637bfa7c-ee64-4429-8726-b0281760e30b","Type":"ContainerStarted","Data":"4d6cc36175bc5802f2a8bab80a15f19b265e6005e3fe0e7ec7287a16d6b9eba1"}
Jul 17 13:50:14.659673 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:14.659517    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"
Jul 17 13:50:14.664197 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:14.664174    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5"
Jul 17 13:50:14.676788 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:14.676750    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-nkvs5" podStartSLOduration=2.787220694 podStartE2EDuration="3.676737867s" totalImagesPullingTime="889.517173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:50:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:14.676378627 +0000 UTC m=+300.382825462" watchObservedRunningTime="2026-07-17 13:50:14.676737867 +0000 UTC m=+300.383184703"
Jul 17 13:50:16.635742 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:16.635709    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8r6d5"
Jul 17 13:50:22.610434 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.610399    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"]
Jul 17 13:50:22.614110 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.614091    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.617036 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.616999    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 17 13:50:22.617158 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.616999    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 13:50:22.617601 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.617578    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 13:50:22.617737 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.617714    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 17 13:50:22.617796 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.617773    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 17 13:50:22.617993 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.617978    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 13:50:22.618134 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.618115    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bh2q7\""
Jul 17 13:50:22.632049 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.632020    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"]
Jul 17 13:50:22.650226 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.650199    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dhxg7"]
Jul 17 13:50:22.653179 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.653162    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.655718 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.655700    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 13:50:22.655827 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.655770    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bbfks\""
Jul 17 13:50:22.656071 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.656051    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 13:50:22.656300 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.656282    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 13:50:22.691883 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.691856    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.692052 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.691927    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.692052 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.691972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thj4s\" (UniqueName: \"kubernetes.io/projected/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-api-access-thj4s\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.692149 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.692062    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.692149 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.692103    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.692149 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.692126    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.793116 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793085    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.793303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793135    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.793303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793153    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa13677c-2c63-4780-8ed8-9de458d822da-metrics-client-ca\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.793303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793172    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thj4s\" (UniqueName: \"kubernetes.io/projected/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-api-access-thj4s\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.793303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793197    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msvfk\" (UniqueName: \"kubernetes.io/projected/aa13677c-2c63-4780-8ed8-9de458d822da-kube-api-access-msvfk\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.793303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793226    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-wtmp\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.793303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793252    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.793303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793283    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.793303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793303    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.793755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793333    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-textfile\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.793755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793358    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-accelerators-collector-config\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.793755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793383    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa13677c-2c63-4780-8ed8-9de458d822da-sys\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.793755 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:22.793404    2561 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 17 13:50:22.793755 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:22.793466    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-tls podName:d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf nodeName:}" failed. No retries permitted until 2026-07-17 13:50:23.293446017 +0000 UTC m=+308.999892829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-bdfvl" (UID: "d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf") : secret "kube-state-metrics-tls" not found
Jul 17 13:50:22.793755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793410    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aa13677c-2c63-4780-8ed8-9de458d822da-root\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.793755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793524    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.793755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793559    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-tls\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.794198 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.793805    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.794382 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.794361    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.794602 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.794579    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.795984 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.795958    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.804932 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.804912    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thj4s\" (UniqueName: \"kubernetes.io/projected/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-api-access-thj4s\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:22.894600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.894520    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.894600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.894571    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa13677c-2c63-4780-8ed8-9de458d822da-metrics-client-ca\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.894600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.894588    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msvfk\" (UniqueName: \"kubernetes.io/projected/aa13677c-2c63-4780-8ed8-9de458d822da-kube-api-access-msvfk\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.894869 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.894608    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-wtmp\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.894869 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.894660    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-textfile\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.894869 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.894688    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-accelerators-collector-config\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.894869 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.894713    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa13677c-2c63-4780-8ed8-9de458d822da-sys\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.894869 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.894742    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aa13677c-2c63-4780-8ed8-9de458d822da-root\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.894869 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.894783    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-tls\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.895193 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.895047    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa13677c-2c63-4780-8ed8-9de458d822da-sys\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.895193 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.895087    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aa13677c-2c63-4780-8ed8-9de458d822da-root\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.895193 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.895107    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-wtmp\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.895348 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.895326    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-textfile\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.895674 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.895653    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-accelerators-collector-config\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.895793 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.895705    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa13677c-2c63-4780-8ed8-9de458d822da-metrics-client-ca\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.897268 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.897238    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-tls\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.897474 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.897452    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa13677c-2c63-4780-8ed8-9de458d822da-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.905715 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.905691    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msvfk\" (UniqueName: \"kubernetes.io/projected/aa13677c-2c63-4780-8ed8-9de458d822da-kube-api-access-msvfk\") pod \"node-exporter-dhxg7\" (UID: \"aa13677c-2c63-4780-8ed8-9de458d822da\") " pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.963179 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.963156    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dhxg7"
Jul 17 13:50:22.975749 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:22.975726    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa13677c_2c63_4780_8ed8_9de458d822da.slice/crio-f5958cf0f7074120f462080acef414aa026e2f7e6db5ccd259d8bfa2a165286c WatchSource:0}: Error finding container f5958cf0f7074120f462080acef414aa026e2f7e6db5ccd259d8bfa2a165286c: Status 404 returned error can't find the container with id f5958cf0f7074120f462080acef414aa026e2f7e6db5ccd259d8bfa2a165286c
Jul 17 13:50:22.977541 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:22.977528    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 13:50:23.297332 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.297301    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:23.299803 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.299781    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-bdfvl\" (UID: \"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:23.525022 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.524980    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"
Jul 17 13:50:23.669252 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.669175    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-bdfvl"]
Jul 17 13:50:23.672127 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:23.672098    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd802ba2c_8c2f_43f3_b427_6cc29a4b7fcf.slice/crio-5c77eb46f8b6934932f2552372a5a7cdfcf2e7a9fae8a68973586ee0ec341270 WatchSource:0}: Error finding container 5c77eb46f8b6934932f2552372a5a7cdfcf2e7a9fae8a68973586ee0ec341270: Status 404 returned error can't find the container with id 5c77eb46f8b6934932f2552372a5a7cdfcf2e7a9fae8a68973586ee0ec341270
Jul 17 13:50:23.689747 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.689694    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl" event={"ID":"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf","Type":"ContainerStarted","Data":"5c77eb46f8b6934932f2552372a5a7cdfcf2e7a9fae8a68973586ee0ec341270"}
Jul 17 13:50:23.691100 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.691063    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dhxg7" event={"ID":"aa13677c-2c63-4780-8ed8-9de458d822da","Type":"ContainerStarted","Data":"f5958cf0f7074120f462080acef414aa026e2f7e6db5ccd259d8bfa2a165286c"}
Jul 17 13:50:23.694204 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.694180    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 13:50:23.699936 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.699921    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.703772 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.703750    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jk946\""
Jul 17 13:50:23.704050 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.704032    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 17 13:50:23.704184 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.704159    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 17 13:50:23.704284 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.704169    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 17 13:50:23.704284 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.704249    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 17 13:50:23.704284 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.704259    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 17 13:50:23.704518 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.704505    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 17 13:50:23.704585 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.704519    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 17 13:50:23.704633 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.704581    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 17 13:50:23.704684 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.704674    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 17 13:50:23.714532 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.714511    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 13:50:23.801063 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801029    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801138    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-web-config\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801175    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801224    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801254    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801281    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs5rc\" (UniqueName: \"kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-kube-api-access-cs5rc\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801314    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-config-out\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801339    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801682 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801418    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801682 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801467    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801682 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801508    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801682 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801553    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.801682 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.801580    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.902782 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.902703    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.902782 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.902750    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.902994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.902786    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.902994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.902805    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.902994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.902822    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.902994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.902853    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.902994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.902890    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-web-config\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.902994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.902914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.902994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.902943    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.902994 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.902975    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.903408 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.903022    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs5rc\" (UniqueName: \"kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-kube-api-access-cs5rc\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.903408 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:23.903057    2561 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 17 13:50:23.903408 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:23.903141    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls podName:1f947df8-6cde-4cb6-a32e-032521497584 nodeName:}" failed. No retries permitted until 2026-07-17 13:50:24.40311831 +0000 UTC m=+310.109565146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1f947df8-6cde-4cb6-a32e-032521497584") : secret "alertmanager-main-tls" not found
Jul 17 13:50:23.903759 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.903061    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-config-out\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.903759 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.903641    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.903759 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.903743    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.903924 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.903809    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.904520 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.904494    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.905948 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.905909    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.907478 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.907436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.907671 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.907621    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.907902 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.907884    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-config-out\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.909966 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.909945    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.910079 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.910032    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.910079 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.910054    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.911609 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.911590    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-web-config\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:23.912980 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:23.912958    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs5rc\" (UniqueName: \"kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-kube-api-access-cs5rc\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:24.409245 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:24.409208    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:24.409416 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:24.409383    2561 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 17 13:50:24.409476 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:24.409449    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls podName:1f947df8-6cde-4cb6-a32e-032521497584 nodeName:}" failed. No retries permitted until 2026-07-17 13:50:25.409434985 +0000 UTC m=+311.115881798 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1f947df8-6cde-4cb6-a32e-032521497584") : secret "alertmanager-main-tls" not found
Jul 17 13:50:24.696333 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:24.696301    2561 generic.go:356] "Generic (PLEG): container finished" podID="aa13677c-2c63-4780-8ed8-9de458d822da" containerID="b624bd896367a3bfdf0b7648f1221acc11a45daf675503ac92e8690234ba623b" exitCode=0
Jul 17 13:50:24.696799 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:24.696355    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dhxg7" event={"ID":"aa13677c-2c63-4780-8ed8-9de458d822da","Type":"ContainerDied","Data":"b624bd896367a3bfdf0b7648f1221acc11a45daf675503ac92e8690234ba623b"}
Jul 17 13:50:24.768563 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:24.768535    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:50:25.416552 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.416513    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:25.419055 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.419034    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:25.510644 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.510613    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:50:25.664075 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.660770    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 13:50:25.667398 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:25.667355    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f947df8_6cde_4cb6_a32e_032521497584.slice/crio-544067c3108c3ff4396a0b25d565d9d78a3ca07a3fc15ba646aa6a061dc0bb73 WatchSource:0}: Error finding container 544067c3108c3ff4396a0b25d565d9d78a3ca07a3fc15ba646aa6a061dc0bb73: Status 404 returned error can't find the container with id 544067c3108c3ff4396a0b25d565d9d78a3ca07a3fc15ba646aa6a061dc0bb73
Jul 17 13:50:25.701733 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.701700    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerStarted","Data":"544067c3108c3ff4396a0b25d565d9d78a3ca07a3fc15ba646aa6a061dc0bb73"}
Jul 17 13:50:25.703970 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.703938    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl" event={"ID":"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf","Type":"ContainerStarted","Data":"185039ff4e56ce6df5c516427c054e05412e60bb6ffc3e91b3e43f0c059a3ead"}
Jul 17 13:50:25.704094 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.703995    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl" event={"ID":"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf","Type":"ContainerStarted","Data":"6d746979e8887fea749ef3b5775da4a44dc70db597823790598fa45adb0a303e"}
Jul 17 13:50:25.704094 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.704051    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl" event={"ID":"d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf","Type":"ContainerStarted","Data":"2dc076437e421813e0d23d49236f98b85e0cda5b958b03feb853cb9618c8343c"}
Jul 17 13:50:25.706031 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.705987    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dhxg7" event={"ID":"aa13677c-2c63-4780-8ed8-9de458d822da","Type":"ContainerStarted","Data":"9bb4ed51c5276c8a60386213f88f53b12749afed911c928bc4521c97a086d6b7"}
Jul 17 13:50:25.706111 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.706048    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dhxg7" event={"ID":"aa13677c-2c63-4780-8ed8-9de458d822da","Type":"ContainerStarted","Data":"f2f665fc3bf624939636ce31c5505b9336b0a8db38907f653d2ae9d16b765f65"}
Jul 17 13:50:25.723854 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.723808    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-bdfvl" podStartSLOduration=2.352826608 podStartE2EDuration="3.723795119s" totalImagesPullingTime="1.370968511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:25.722786225 +0000 UTC m=+311.429233060" watchObservedRunningTime="2026-07-17 13:50:25.723795119 +0000 UTC m=+311.430241952"
Jul 17 13:50:25.745616 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:25.745578    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dhxg7" podStartSLOduration=2.690622407 podStartE2EDuration="3.745565642s" totalImagesPullingTime="986.220854ms" totalInitContainerRuntime="68.722381ms" isStatefulPod=true podCreationTimestamp="2026-07-17 13:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:25.74469789 +0000 UTC m=+311.451144723" watchObservedRunningTime="2026-07-17 13:50:25.745565642 +0000 UTC m=+311.452012475"
Jul 17 13:50:26.936062 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:26.936038    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-69f8948bd-j5gn8"]
Jul 17 13:50:26.940144 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:26.940128    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:26.942634 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:26.942611    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 17 13:50:26.942839 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:26.942802    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 17 13:50:26.943673 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:26.943652    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-d8psf\""
Jul 17 13:50:26.943771 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:26.943674    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 17 13:50:26.943771 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:26.943694    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1k4trd1sbrivb\""
Jul 17 13:50:26.943886 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:26.943682    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 17 13:50:26.950751 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:26.950728    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69f8948bd-j5gn8"]
Jul 17 13:50:27.032739 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.032716    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/18bbb18d-fb5b-4a33-856e-11badc504f7b-secret-metrics-server-client-certs\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.032849 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.032751    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18bbb18d-fb5b-4a33-856e-11badc504f7b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.032849 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.032771    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/18bbb18d-fb5b-4a33-856e-11badc504f7b-metrics-server-audit-profiles\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.032849 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.032797    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18bbb18d-fb5b-4a33-856e-11badc504f7b-client-ca-bundle\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.032968 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.032852    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/18bbb18d-fb5b-4a33-856e-11badc504f7b-secret-metrics-server-tls\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.032968 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.032874    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmt4t\" (UniqueName: \"kubernetes.io/projected/18bbb18d-fb5b-4a33-856e-11badc504f7b-kube-api-access-pmt4t\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.032968 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.032909    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/18bbb18d-fb5b-4a33-856e-11badc504f7b-audit-log\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.133763 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.133736    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/18bbb18d-fb5b-4a33-856e-11badc504f7b-secret-metrics-server-tls\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.133859 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.133765    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmt4t\" (UniqueName: \"kubernetes.io/projected/18bbb18d-fb5b-4a33-856e-11badc504f7b-kube-api-access-pmt4t\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.133859 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.133785    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/18bbb18d-fb5b-4a33-856e-11badc504f7b-audit-log\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.133859 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.133837    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/18bbb18d-fb5b-4a33-856e-11badc504f7b-secret-metrics-server-client-certs\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.134042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.133859    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18bbb18d-fb5b-4a33-856e-11badc504f7b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.134042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.133877    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/18bbb18d-fb5b-4a33-856e-11badc504f7b-metrics-server-audit-profiles\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.134042 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.133897    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18bbb18d-fb5b-4a33-856e-11badc504f7b-client-ca-bundle\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.134314 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.134295    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/18bbb18d-fb5b-4a33-856e-11badc504f7b-audit-log\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.135075 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.135052    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18bbb18d-fb5b-4a33-856e-11badc504f7b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.135226 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.135203    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/18bbb18d-fb5b-4a33-856e-11badc504f7b-metrics-server-audit-profiles\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.136306 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.136280    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/18bbb18d-fb5b-4a33-856e-11badc504f7b-secret-metrics-server-client-certs\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.136481 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.136461    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/18bbb18d-fb5b-4a33-856e-11badc504f7b-secret-metrics-server-tls\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.136521 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.136497    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18bbb18d-fb5b-4a33-856e-11badc504f7b-client-ca-bundle\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.143483 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.143465    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmt4t\" (UniqueName: \"kubernetes.io/projected/18bbb18d-fb5b-4a33-856e-11badc504f7b-kube-api-access-pmt4t\") pod \"metrics-server-69f8948bd-j5gn8\" (UID: \"18bbb18d-fb5b-4a33-856e-11badc504f7b\") " pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.250634 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.250611    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:27.388844 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.388812    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"]
Jul 17 13:50:27.393183 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.393169    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:27.395905 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.395796    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 17 13:50:27.395905 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.395862    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69f8948bd-j5gn8"]
Jul 17 13:50:27.395905 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.395893    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-lqtxd\""
Jul 17 13:50:27.401345 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:27.400946    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18bbb18d_fb5b_4a33_856e_11badc504f7b.slice/crio-3428d2a107680188199cf0d0c51f9fd8b911fa34a7ef6c3456eea6b6b48e7c3d WatchSource:0}: Error finding container 3428d2a107680188199cf0d0c51f9fd8b911fa34a7ef6c3456eea6b6b48e7c3d: Status 404 returned error can't find the container with id 3428d2a107680188199cf0d0c51f9fd8b911fa34a7ef6c3456eea6b6b48e7c3d
Jul 17 13:50:27.401847 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.401828    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"]
Jul 17 13:50:27.538601 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.538521    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98716b2a-fe1e-421a-8b56-a4257aaf3bef-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5nc7j\" (UID: \"98716b2a-fe1e-421a-8b56-a4257aaf3bef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:27.538601 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.538568    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbr9x\" (UniqueName: \"kubernetes.io/projected/98716b2a-fe1e-421a-8b56-a4257aaf3bef-kube-api-access-lbr9x\") pod \"monitoring-plugin-7b7b7bc8bf-5nc7j\" (UID: \"98716b2a-fe1e-421a-8b56-a4257aaf3bef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:27.639249 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.639210    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98716b2a-fe1e-421a-8b56-a4257aaf3bef-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5nc7j\" (UID: \"98716b2a-fe1e-421a-8b56-a4257aaf3bef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:27.639414 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.639304    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbr9x\" (UniqueName: \"kubernetes.io/projected/98716b2a-fe1e-421a-8b56-a4257aaf3bef-kube-api-access-lbr9x\") pod \"monitoring-plugin-7b7b7bc8bf-5nc7j\" (UID: \"98716b2a-fe1e-421a-8b56-a4257aaf3bef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:27.639414 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:27.639354    2561 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 17 13:50:27.639485 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:50:27.639417    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98716b2a-fe1e-421a-8b56-a4257aaf3bef-monitoring-plugin-cert podName:98716b2a-fe1e-421a-8b56-a4257aaf3bef nodeName:}" failed. No retries permitted until 2026-07-17 13:50:28.139401103 +0000 UTC m=+313.845847918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/98716b2a-fe1e-421a-8b56-a4257aaf3bef-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-5nc7j" (UID: "98716b2a-fe1e-421a-8b56-a4257aaf3bef") : secret "monitoring-plugin-cert" not found
Jul 17 13:50:27.650696 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.650671    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbr9x\" (UniqueName: \"kubernetes.io/projected/98716b2a-fe1e-421a-8b56-a4257aaf3bef-kube-api-access-lbr9x\") pod \"monitoring-plugin-7b7b7bc8bf-5nc7j\" (UID: \"98716b2a-fe1e-421a-8b56-a4257aaf3bef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:27.715070 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.715040    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8" event={"ID":"18bbb18d-fb5b-4a33-856e-11badc504f7b","Type":"ContainerStarted","Data":"3428d2a107680188199cf0d0c51f9fd8b911fa34a7ef6c3456eea6b6b48e7c3d"}
Jul 17 13:50:27.716556 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.716530    2561 generic.go:356] "Generic (PLEG): container finished" podID="1f947df8-6cde-4cb6-a32e-032521497584" containerID="ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46" exitCode=0
Jul 17 13:50:27.716689 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.716603    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerDied","Data":"ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46"}
Jul 17 13:50:27.873681 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.873600    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"]
Jul 17 13:50:27.878034 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.878017    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:27.881680 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.881657    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 17 13:50:27.881818 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.881730    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-b68fd\""
Jul 17 13:50:27.881899 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.881884    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 17 13:50:27.882072 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.882058    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 17 13:50:27.882365 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.882346    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 17 13:50:27.882603 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.882579    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 17 13:50:27.887710 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.887692    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 17 13:50:27.894128 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:27.894110    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"]
Jul 17 13:50:28.044507 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.044469    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65051f9e-1a46-40b3-8c44-45437d21f9c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.044924 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.044562    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmfg2\" (UniqueName: \"kubernetes.io/projected/65051f9e-1a46-40b3-8c44-45437d21f9c8-kube-api-access-cmfg2\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.044924 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.044596    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-federate-client-tls\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.044924 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.044689    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-telemeter-client-tls\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.044924 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.044738    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.044924 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.044770    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-secret-telemeter-client\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.044924 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.044822    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65051f9e-1a46-40b3-8c44-45437d21f9c8-metrics-client-ca\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.044924 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.044886    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65051f9e-1a46-40b3-8c44-45437d21f9c8-serving-certs-ca-bundle\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.146344 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.146278    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65051f9e-1a46-40b3-8c44-45437d21f9c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.146486 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.146354    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmfg2\" (UniqueName: \"kubernetes.io/projected/65051f9e-1a46-40b3-8c44-45437d21f9c8-kube-api-access-cmfg2\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.146486 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.146388    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-federate-client-tls\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.146486 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.146425    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98716b2a-fe1e-421a-8b56-a4257aaf3bef-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5nc7j\" (UID: \"98716b2a-fe1e-421a-8b56-a4257aaf3bef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:28.146486 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.146471    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-telemeter-client-tls\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.146680 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.146521    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.146680 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.146551    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-secret-telemeter-client\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.146680 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.146581    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65051f9e-1a46-40b3-8c44-45437d21f9c8-metrics-client-ca\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.146680 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.146624    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65051f9e-1a46-40b3-8c44-45437d21f9c8-serving-certs-ca-bundle\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.147298 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.147248    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65051f9e-1a46-40b3-8c44-45437d21f9c8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.147598 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.147302    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65051f9e-1a46-40b3-8c44-45437d21f9c8-serving-certs-ca-bundle\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.147890 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.147852    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65051f9e-1a46-40b3-8c44-45437d21f9c8-metrics-client-ca\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.149745 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.149709    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.150333 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.150290    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-telemeter-client-tls\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.150333 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.150314    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-federate-client-tls\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.150670 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.150651    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/65051f9e-1a46-40b3-8c44-45437d21f9c8-secret-telemeter-client\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.150995 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.150968    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98716b2a-fe1e-421a-8b56-a4257aaf3bef-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-5nc7j\" (UID: \"98716b2a-fe1e-421a-8b56-a4257aaf3bef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:28.159502 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.159462    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmfg2\" (UniqueName: \"kubernetes.io/projected/65051f9e-1a46-40b3-8c44-45437d21f9c8-kube-api-access-cmfg2\") pod \"telemeter-client-794ddcf88c-fmpd8\" (UID: \"65051f9e-1a46-40b3-8c44-45437d21f9c8\") " pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.188220 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.188191    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"
Jul 17 13:50:28.307246 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.307198    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:28.376394 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.376141    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-794ddcf88c-fmpd8"]
Jul 17 13:50:28.379456 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:28.379407    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65051f9e_1a46_40b3_8c44_45437d21f9c8.slice/crio-ce6fd3f4616bec085083b8f42ba94a7c51bb58c5fe2969c731467d1d877dc6e9 WatchSource:0}: Error finding container ce6fd3f4616bec085083b8f42ba94a7c51bb58c5fe2969c731467d1d877dc6e9: Status 404 returned error can't find the container with id ce6fd3f4616bec085083b8f42ba94a7c51bb58c5fe2969c731467d1d877dc6e9
Jul 17 13:50:28.471766 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.471685    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"]
Jul 17 13:50:28.475065 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:28.475040    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98716b2a_fe1e_421a_8b56_a4257aaf3bef.slice/crio-9d3d47f4cfdcb9ed56535e11e3f47019393d0c89db4165f284a9ab7373b05783 WatchSource:0}: Error finding container 9d3d47f4cfdcb9ed56535e11e3f47019393d0c89db4165f284a9ab7373b05783: Status 404 returned error can't find the container with id 9d3d47f4cfdcb9ed56535e11e3f47019393d0c89db4165f284a9ab7373b05783
Jul 17 13:50:28.724893 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.724823    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j" event={"ID":"98716b2a-fe1e-421a-8b56-a4257aaf3bef","Type":"ContainerStarted","Data":"9d3d47f4cfdcb9ed56535e11e3f47019393d0c89db4165f284a9ab7373b05783"}
Jul 17 13:50:28.726381 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:28.726350    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8" event={"ID":"65051f9e-1a46-40b3-8c44-45437d21f9c8","Type":"ContainerStarted","Data":"ce6fd3f4616bec085083b8f42ba94a7c51bb58c5fe2969c731467d1d877dc6e9"}
Jul 17 13:50:29.648709 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:29.648433    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d879b7597-vdcs5"
Jul 17 13:50:29.733329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:29.733285    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8" event={"ID":"18bbb18d-fb5b-4a33-856e-11badc504f7b","Type":"ContainerStarted","Data":"54822d6615faeac78e1b351f3a5e8d6e6773d923e5fb2dee05dbdc6ec07d9ba5"}
Jul 17 13:50:29.737137 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:29.737105    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerStarted","Data":"2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65"}
Jul 17 13:50:29.737273 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:29.737180    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerStarted","Data":"2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1"}
Jul 17 13:50:29.737273 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:29.737215    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerStarted","Data":"66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb"}
Jul 17 13:50:30.746069 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:30.746030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8" event={"ID":"65051f9e-1a46-40b3-8c44-45437d21f9c8","Type":"ContainerStarted","Data":"4fde3b2ba294e25882221e249bb31fac7d02b0d28d86b2d5bed1ad5256711307"}
Jul 17 13:50:30.746452 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:30.746141    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8" event={"ID":"65051f9e-1a46-40b3-8c44-45437d21f9c8","Type":"ContainerStarted","Data":"c0a64fea059e1e9ec2422bae96e429c9557d7d988a7f9cc869d1d2a99790df67"}
Jul 17 13:50:30.751891 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:30.751767    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerStarted","Data":"2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7"}
Jul 17 13:50:30.751891 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:30.751836    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerStarted","Data":"723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895"}
Jul 17 13:50:30.756746 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:30.756412    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j" event={"ID":"98716b2a-fe1e-421a-8b56-a4257aaf3bef","Type":"ContainerStarted","Data":"70ff7f4088969a20aa713683505309b8d4793d1b4f9b303e0ec3446cde56cdf6"}
Jul 17 13:50:30.756746 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:30.756714    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:30.762240 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:30.762222    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j"
Jul 17 13:50:30.777231 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:30.777186    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-5nc7j" podStartSLOduration=1.680152014 podStartE2EDuration="3.777170519s" totalImagesPullingTime="2.097018505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:50:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:30.776235299 +0000 UTC m=+316.482682148" watchObservedRunningTime="2026-07-17 13:50:30.777170519 +0000 UTC m=+316.483617353"
Jul 17 13:50:30.778339 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:30.778303    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8" podStartSLOduration=2.775701039 podStartE2EDuration="4.778290452s" totalImagesPullingTime="2.002589413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:29.760361851 +0000 UTC m=+315.466808687" watchObservedRunningTime="2026-07-17 13:50:30.778290452 +0000 UTC m=+316.484737285"
Jul 17 13:50:31.765452 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:31.765420    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerStarted","Data":"6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e"}
Jul 17 13:50:31.767418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:31.767396    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8" event={"ID":"65051f9e-1a46-40b3-8c44-45437d21f9c8","Type":"ContainerStarted","Data":"d83fbe882a71604498c4094c8e6d836c1bb099bf3d7eb4b5c4c0ae3a78ca5acb"}
Jul 17 13:50:31.803241 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:31.803187    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.174884412 podStartE2EDuration="8.803170271s" totalImagesPullingTime="3.61170695s" totalInitContainerRuntime="16.578909ms" isStatefulPod=false podCreationTimestamp="2026-07-17 13:50:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:31.80077065 +0000 UTC m=+317.507217505" watchObservedRunningTime="2026-07-17 13:50:31.803170271 +0000 UTC m=+317.509617116"
Jul 17 13:50:31.830950 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:31.830901    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-794ddcf88c-fmpd8" podStartSLOduration=2.637306502 podStartE2EDuration="4.830884236s" totalImagesPullingTime="2.193577734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:50:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:31.830298369 +0000 UTC m=+317.536745204" watchObservedRunningTime="2026-07-17 13:50:31.830884236 +0000 UTC m=+317.537331072"
Jul 17 13:50:36.870439 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:36.870394    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-w66vp"]
Jul 17 13:50:36.947759 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:36.947712    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-w66vp"]
Jul 17 13:50:36.947899 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:36.947825    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-w66vp"
Jul 17 13:50:36.950928 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:36.950894    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 13:50:36.951120 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:36.950965    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-qgjbv\""
Jul 17 13:50:36.951120 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:36.950991    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 13:50:37.022826 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:37.022785    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cwlv\" (UniqueName: \"kubernetes.io/projected/d38007da-cfed-4ff5-9d29-ce756630ec75-kube-api-access-8cwlv\") pod \"downloads-5bc67dcd46-w66vp\" (UID: \"d38007da-cfed-4ff5-9d29-ce756630ec75\") " pod="openshift-console/downloads-5bc67dcd46-w66vp"
Jul 17 13:50:37.123948 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:37.123867    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cwlv\" (UniqueName: \"kubernetes.io/projected/d38007da-cfed-4ff5-9d29-ce756630ec75-kube-api-access-8cwlv\") pod \"downloads-5bc67dcd46-w66vp\" (UID: \"d38007da-cfed-4ff5-9d29-ce756630ec75\") " pod="openshift-console/downloads-5bc67dcd46-w66vp"
Jul 17 13:50:37.132907 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:37.132877    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cwlv\" (UniqueName: \"kubernetes.io/projected/d38007da-cfed-4ff5-9d29-ce756630ec75-kube-api-access-8cwlv\") pod \"downloads-5bc67dcd46-w66vp\" (UID: \"d38007da-cfed-4ff5-9d29-ce756630ec75\") " pod="openshift-console/downloads-5bc67dcd46-w66vp"
Jul 17 13:50:37.257328 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:37.257288    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-w66vp"
Jul 17 13:50:37.450557 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:37.450525    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-w66vp"]
Jul 17 13:50:37.453605 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:37.453573    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38007da_cfed_4ff5_9d29_ce756630ec75.slice/crio-efb9cf0af197e9e8d419e97a5cb5fa99972508bcc11a27b35e609ffd4ce88bad WatchSource:0}: Error finding container efb9cf0af197e9e8d419e97a5cb5fa99972508bcc11a27b35e609ffd4ce88bad: Status 404 returned error can't find the container with id efb9cf0af197e9e8d419e97a5cb5fa99972508bcc11a27b35e609ffd4ce88bad
Jul 17 13:50:37.789164 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:37.789125    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-w66vp" event={"ID":"d38007da-cfed-4ff5-9d29-ce756630ec75","Type":"ContainerStarted","Data":"efb9cf0af197e9e8d419e97a5cb5fa99972508bcc11a27b35e609ffd4ce88bad"}
Jul 17 13:50:42.468396 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.468363    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78cf49f676-8lqgn"]
Jul 17 13:50:42.603207 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.603174    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78cf49f676-8lqgn"]
Jul 17 13:50:42.603378 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.603285    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.605914 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.605696    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 13:50:42.605914 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.605767    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 13:50:42.605914 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.605814    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 13:50:42.606262 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.606061    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gxmth\""
Jul 17 13:50:42.606790 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.606759    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 13:50:42.606885 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.606828    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 13:50:42.675810 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.675779    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-service-ca\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.675966 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.675818    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-serving-cert\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.675966 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.675886    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-config\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.675966 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.675955    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-oauth-config\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.676163 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.675985    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-oauth-serving-cert\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.676163 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.676086    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcj67\" (UniqueName: \"kubernetes.io/projected/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-kube-api-access-kcj67\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.777352 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.777282    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcj67\" (UniqueName: \"kubernetes.io/projected/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-kube-api-access-kcj67\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.777499 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.777368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-service-ca\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.777499 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.777393    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-serving-cert\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.777499 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.777437    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-config\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.777499 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.777487    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-oauth-config\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.777721 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.777509    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-oauth-serving-cert\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.778169 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.778146    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-config\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.778266 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.778199    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-service-ca\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.778266 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.778209    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-oauth-serving-cert\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.780775 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.780752    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-oauth-config\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.780936 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.780893    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-serving-cert\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.786893 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.786866    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcj67\" (UniqueName: \"kubernetes.io/projected/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-kube-api-access-kcj67\") pod \"console-78cf49f676-8lqgn\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") " pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:42.914308 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:42.914272    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:43.071080 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:43.070941    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78cf49f676-8lqgn"]
Jul 17 13:50:43.075475 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:43.075448    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdbbb9a9_63de_4464_9879_475b9e6cd2e2.slice/crio-ac338f300beb34cea3635e5511db5ce35a48981995ca61a35603a43266ff1654 WatchSource:0}: Error finding container ac338f300beb34cea3635e5511db5ce35a48981995ca61a35603a43266ff1654: Status 404 returned error can't find the container with id ac338f300beb34cea3635e5511db5ce35a48981995ca61a35603a43266ff1654
Jul 17 13:50:43.819352 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:43.819314    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78cf49f676-8lqgn" event={"ID":"bdbbb9a9-63de-4464-9879-475b9e6cd2e2","Type":"ContainerStarted","Data":"ac338f300beb34cea3635e5511db5ce35a48981995ca61a35603a43266ff1654"}
Jul 17 13:50:46.836043 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:46.835943    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78cf49f676-8lqgn" event={"ID":"bdbbb9a9-63de-4464-9879-475b9e6cd2e2","Type":"ContainerStarted","Data":"979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4"}
Jul 17 13:50:46.855190 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:46.855141    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78cf49f676-8lqgn" podStartSLOduration=1.431705223 podStartE2EDuration="4.855125382s" totalImagesPullingTime="3.423420159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:50:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:46.854041754 +0000 UTC m=+332.560488590" watchObservedRunningTime="2026-07-17 13:50:46.855125382 +0000 UTC m=+332.561572216"
Jul 17 13:50:47.251641 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:47.251612    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:47.251824 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:47.251675    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:50:51.181273 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.181239    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f75bf7855-kvffm"]
Jul 17 13:50:51.183720 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.183699    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.192406 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.192379    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 17 13:50:51.197538 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.197482    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f75bf7855-kvffm"]
Jul 17 13:50:51.257113 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.257061    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gffs5\" (UniqueName: \"kubernetes.io/projected/62494c12-5cdf-4847-920b-0f125a4db466-kube-api-access-gffs5\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.257113 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.257100    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-service-ca\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.257350 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.257141    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-serving-cert\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.257350 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.257194    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-oauth-serving-cert\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.257350 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.257295    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-trusted-ca-bundle\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.257350 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.257333    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-console-config\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.257517 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.257366    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-oauth-config\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.358542 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.358508    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gffs5\" (UniqueName: \"kubernetes.io/projected/62494c12-5cdf-4847-920b-0f125a4db466-kube-api-access-gffs5\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.358542 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.358544    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-service-ca\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.358760 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.358567    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-serving-cert\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.358760 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.358600    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-oauth-serving-cert\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.358760 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.358666    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-trusted-ca-bundle\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.358760 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.358691    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-console-config\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.358977 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.358949    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-oauth-config\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.359378 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.359351    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-oauth-serving-cert\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.359506 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.359351    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-service-ca\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.359506 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.359415    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-console-config\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.359638 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.359534    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-trusted-ca-bundle\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.361591 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.361565    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-serving-cert\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.361591 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.361581    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-oauth-config\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.367373 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.367354    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gffs5\" (UniqueName: \"kubernetes.io/projected/62494c12-5cdf-4847-920b-0f125a4db466-kube-api-access-gffs5\") pod \"console-f75bf7855-kvffm\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") " pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:51.495909 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:51.495825    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:50:52.914420 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:52.914386    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:52.914836 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:52.914451    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:52.920772 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:52.920750    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:53.871876 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:53.871845    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:50:55.832370 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:55.832109    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f75bf7855-kvffm"]
Jul 17 13:50:55.834037 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:50:55.833994    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62494c12_5cdf_4847_920b_0f125a4db466.slice/crio-e44f6093e0af9595de139ab8d0b7fa80ea863a6a5338c849f0132135ffb44937 WatchSource:0}: Error finding container e44f6093e0af9595de139ab8d0b7fa80ea863a6a5338c849f0132135ffb44937: Status 404 returned error can't find the container with id e44f6093e0af9595de139ab8d0b7fa80ea863a6a5338c849f0132135ffb44937
Jul 17 13:50:55.875629 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:55.875597    2561 generic.go:356] "Generic (PLEG): container finished" podID="a04aa92c-47f8-4dd2-bc0f-cb8604321dc8" containerID="b88cf9862291a1d825f88bbda4345ae3ba735a0b2974e0c5c8cf1aaf57183204" exitCode=0
Jul 17 13:50:55.875739 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:55.875635    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-kp8xd" event={"ID":"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8","Type":"ContainerDied","Data":"b88cf9862291a1d825f88bbda4345ae3ba735a0b2974e0c5c8cf1aaf57183204"}
Jul 17 13:50:55.876060 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:55.876036    2561 scope.go:117] "RemoveContainer" containerID="b88cf9862291a1d825f88bbda4345ae3ba735a0b2974e0c5c8cf1aaf57183204"
Jul 17 13:50:55.877616 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:55.877589    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-w66vp" event={"ID":"d38007da-cfed-4ff5-9d29-ce756630ec75","Type":"ContainerStarted","Data":"f95b69937edd8636a270bf44e7770f2c59e5544057549f664937c66494780372"}
Jul 17 13:50:55.877795 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:55.877753    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-w66vp"
Jul 17 13:50:55.879281 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:55.879240    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f75bf7855-kvffm" event={"ID":"62494c12-5cdf-4847-920b-0f125a4db466","Type":"ContainerStarted","Data":"e44f6093e0af9595de139ab8d0b7fa80ea863a6a5338c849f0132135ffb44937"}
Jul 17 13:50:55.879683 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:55.879662    2561 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-w66vp container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.21:8080/\": dial tcp 10.133.0.21:8080: connect: connection refused" start-of-body=
Jul 17 13:50:55.879782 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:55.879706    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-w66vp" podUID="d38007da-cfed-4ff5-9d29-ce756630ec75" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.21:8080/\": dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 17 13:50:55.908806 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:55.908753    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-w66vp" podStartSLOduration=1.5808929090000001 podStartE2EDuration="19.908732496s" totalImagesPullingTime="18.327839587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:55.907247284 +0000 UTC m=+341.613694114" watchObservedRunningTime="2026-07-17 13:50:55.908732496 +0000 UTC m=+341.615179331"
Jul 17 13:50:56.885839 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:56.885796    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f75bf7855-kvffm" event={"ID":"62494c12-5cdf-4847-920b-0f125a4db466","Type":"ContainerStarted","Data":"25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa"}
Jul 17 13:50:56.888132 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:56.888098    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-kp8xd" event={"ID":"a04aa92c-47f8-4dd2-bc0f-cb8604321dc8","Type":"ContainerStarted","Data":"438ddf899f90f26ece2bf4e5a6cc00d5a82f81ddfbb7c03cb52933575e7d0009"}
Jul 17 13:50:56.904597 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:56.904556    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f75bf7855-kvffm" podStartSLOduration=5.904540739 podStartE2EDuration="5.904540739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:50:56.90354352 +0000 UTC m=+342.609990357" watchObservedRunningTime="2026-07-17 13:50:56.904540739 +0000 UTC m=+342.610987573"
Jul 17 13:50:56.905183 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:50:56.905132    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-w66vp"
Jul 17 13:51:01.496282 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:01.496247    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:51:01.496804 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:01.496568    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:51:01.501694 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:01.501670    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:51:01.922052 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:01.922021    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:51:01.971555 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:01.971525    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78cf49f676-8lqgn"]
Jul 17 13:51:05.935796 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:05.935715    2561 generic.go:356] "Generic (PLEG): container finished" podID="6e15a328-f305-4234-af3e-5c3779bbb526" containerID="62250cfc8ade207db014db26821f9928c3192df92300571bbab96f74f6182e19" exitCode=0
Jul 17 13:51:05.935796 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:05.935772    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8" event={"ID":"6e15a328-f305-4234-af3e-5c3779bbb526","Type":"ContainerDied","Data":"62250cfc8ade207db014db26821f9928c3192df92300571bbab96f74f6182e19"}
Jul 17 13:51:05.936266 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:05.936055    2561 scope.go:117] "RemoveContainer" containerID="62250cfc8ade207db014db26821f9928c3192df92300571bbab96f74f6182e19"
Jul 17 13:51:06.942996 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:06.942962    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-w9wg8" event={"ID":"6e15a328-f305-4234-af3e-5c3779bbb526","Type":"ContainerStarted","Data":"db231d137231f8ca9adb200b36bf3dfaf4e68b43de12530abe83c6897c58cf9f"}
Jul 17 13:51:07.257019 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:07.256928    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:51:07.261616 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:07.261594    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-69f8948bd-j5gn8"
Jul 17 13:51:26.998424 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:26.998358    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78cf49f676-8lqgn" podUID="bdbbb9a9-63de-4464-9879-475b9e6cd2e2" containerName="console" containerID="cri-o://979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4" gracePeriod=15
Jul 17 13:51:27.273291 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.273232    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:51:27.368990 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.368958    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcj67\" (UniqueName: \"kubernetes.io/projected/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-kube-api-access-kcj67\") pod \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") "
Jul 17 13:51:27.369136 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.369001    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-serving-cert\") pod \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") "
Jul 17 13:51:27.369136 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.369033    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-config\") pod \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") "
Jul 17 13:51:27.369136 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.369061    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-oauth-config\") pod \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") "
Jul 17 13:51:27.369136 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.369084    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-oauth-serving-cert\") pod \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") "
Jul 17 13:51:27.369136 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.369133    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-service-ca\") pod \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\" (UID: \"bdbbb9a9-63de-4464-9879-475b9e6cd2e2\") "
Jul 17 13:51:27.369560 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.369533    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-config" (OuterVolumeSpecName: "console-config") pod "bdbbb9a9-63de-4464-9879-475b9e6cd2e2" (UID: "bdbbb9a9-63de-4464-9879-475b9e6cd2e2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:51:27.369841 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.369528    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bdbbb9a9-63de-4464-9879-475b9e6cd2e2" (UID: "bdbbb9a9-63de-4464-9879-475b9e6cd2e2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:51:27.369841 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.369599    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-service-ca" (OuterVolumeSpecName: "service-ca") pod "bdbbb9a9-63de-4464-9879-475b9e6cd2e2" (UID: "bdbbb9a9-63de-4464-9879-475b9e6cd2e2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:51:27.371382 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.371363    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bdbbb9a9-63de-4464-9879-475b9e6cd2e2" (UID: "bdbbb9a9-63de-4464-9879-475b9e6cd2e2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:51:27.371481 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.371406    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-kube-api-access-kcj67" (OuterVolumeSpecName: "kube-api-access-kcj67") pod "bdbbb9a9-63de-4464-9879-475b9e6cd2e2" (UID: "bdbbb9a9-63de-4464-9879-475b9e6cd2e2"). InnerVolumeSpecName "kube-api-access-kcj67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:51:27.371481 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.371469    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bdbbb9a9-63de-4464-9879-475b9e6cd2e2" (UID: "bdbbb9a9-63de-4464-9879-475b9e6cd2e2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:51:27.470128 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.470101    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-serving-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:27.470128 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.470126    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-config\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:27.470128 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.470136    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-console-oauth-config\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:27.470321 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.470144    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-oauth-serving-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:27.470321 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.470153    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:27.470321 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:27.470162    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcj67\" (UniqueName: \"kubernetes.io/projected/bdbbb9a9-63de-4464-9879-475b9e6cd2e2-kube-api-access-kcj67\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:28.034225 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.034187    2561 generic.go:356] "Generic (PLEG): container finished" podID="bdbbb9a9-63de-4464-9879-475b9e6cd2e2" containerID="979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4" exitCode=0
Jul 17 13:51:28.034615 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.034233    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78cf49f676-8lqgn"
Jul 17 13:51:28.034615 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.034248    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78cf49f676-8lqgn" event={"ID":"bdbbb9a9-63de-4464-9879-475b9e6cd2e2","Type":"ContainerDied","Data":"979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4"}
Jul 17 13:51:28.034615 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.034296    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78cf49f676-8lqgn" event={"ID":"bdbbb9a9-63de-4464-9879-475b9e6cd2e2","Type":"ContainerDied","Data":"ac338f300beb34cea3635e5511db5ce35a48981995ca61a35603a43266ff1654"}
Jul 17 13:51:28.034615 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.034335    2561 scope.go:117] "RemoveContainer" containerID="979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4"
Jul 17 13:51:28.043285 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.043264    2561 scope.go:117] "RemoveContainer" containerID="979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4"
Jul 17 13:51:28.043549 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:51:28.043531    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4\": container with ID starting with 979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4 not found: ID does not exist" containerID="979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4"
Jul 17 13:51:28.043593 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.043558    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4"} err="failed to get container status \"979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4\": rpc error: code = NotFound desc = could not find container \"979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4\": container with ID starting with 979c590b26077d79432a7feecfb948cf51d40a21dd7a483cc9f9f98911d034a4 not found: ID does not exist"
Jul 17 13:51:28.054907 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.054883    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78cf49f676-8lqgn"]
Jul 17 13:51:28.059101 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.059073    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78cf49f676-8lqgn"]
Jul 17 13:51:28.577925 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.577900    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:51:28.580255 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.580229    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5ffda97-3c84-479c-b0c2-d1c62ddc90d7-metrics-certs\") pod \"network-metrics-daemon-7dbv7\" (UID: \"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7\") " pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:51:28.671960 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.671939    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xnsb\""
Jul 17 13:51:28.679976 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.679962    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7dbv7"
Jul 17 13:51:28.772139 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.772108    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdbbb9a9-63de-4464-9879-475b9e6cd2e2" path="/var/lib/kubelet/pods/bdbbb9a9-63de-4464-9879-475b9e6cd2e2/volumes"
Jul 17 13:51:28.829040 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:28.829016    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7dbv7"]
Jul 17 13:51:28.831379 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:51:28.831352    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ffda97_3c84_479c_b0c2_d1c62ddc90d7.slice/crio-e6126b1c5e465fd8a4be92e97eaeed9f01c67fef1091110682393f140456d568 WatchSource:0}: Error finding container e6126b1c5e465fd8a4be92e97eaeed9f01c67fef1091110682393f140456d568: Status 404 returned error can't find the container with id e6126b1c5e465fd8a4be92e97eaeed9f01c67fef1091110682393f140456d568
Jul 17 13:51:29.040822 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:29.040795    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7dbv7" event={"ID":"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7","Type":"ContainerStarted","Data":"e6126b1c5e465fd8a4be92e97eaeed9f01c67fef1091110682393f140456d568"}
Jul 17 13:51:31.052173 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:31.052131    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7dbv7" event={"ID":"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7","Type":"ContainerStarted","Data":"d2529896d8ca76f5bbbd224bef2aaf5805137008eb8d5e9973ef01cd6fbcd80f"}
Jul 17 13:51:31.052562 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:31.052219    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7dbv7" event={"ID":"c5ffda97-3c84-479c-b0c2-d1c62ddc90d7","Type":"ContainerStarted","Data":"b1caaba06855d476c5ad4140aa2420349bb39b613deab090d99f3dd185d56782"}
Jul 17 13:51:31.070131 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:31.070084    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7dbv7" podStartSLOduration=374.751239961 podStartE2EDuration="6m16.070070221s" totalImagesPullingTime="1.31883026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:51:31.068722769 +0000 UTC m=+376.775169602" watchObservedRunningTime="2026-07-17 13:51:31.070070221 +0000 UTC m=+376.776517055"
Jul 17 13:51:40.850596 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.850561    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c44494d47-7g9f6"]
Jul 17 13:51:40.851059 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.850867    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdbbb9a9-63de-4464-9879-475b9e6cd2e2" containerName="console"
Jul 17 13:51:40.851059 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.850878    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdbbb9a9-63de-4464-9879-475b9e6cd2e2" containerName="console"
Jul 17 13:51:40.851059 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.850940    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdbbb9a9-63de-4464-9879-475b9e6cd2e2" containerName="console"
Jul 17 13:51:40.868524 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.868495    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:40.872171 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.872148    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c44494d47-7g9f6"]
Jul 17 13:51:40.985868 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.985840    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-service-ca\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:40.985868 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.985870    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-trusted-ca-bundle\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:40.986062 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.985897    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-console-config\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:40.986062 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.985982    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-serving-cert\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:40.986062 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.986027    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfspn\" (UniqueName: \"kubernetes.io/projected/80b27150-4ddd-467c-acec-c4fb66953e48-kube-api-access-rfspn\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:40.986062 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.986052    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-oauth-serving-cert\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:40.986213 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:40.986099    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-oauth-config\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.087372 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.087344    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-oauth-config\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.087509 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.087378    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-service-ca\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.087509 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.087393    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-trusted-ca-bundle\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.087609 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.087519    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-console-config\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.087609 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.087590    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-serving-cert\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.087711 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.087616    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfspn\" (UniqueName: \"kubernetes.io/projected/80b27150-4ddd-467c-acec-c4fb66953e48-kube-api-access-rfspn\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.087711 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.087646    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-oauth-serving-cert\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.088296 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.088258    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-service-ca\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.088409 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.088297    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-console-config\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.088409 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.088378    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-oauth-serving-cert\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.088488 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.088407    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-trusted-ca-bundle\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.090671 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.090640    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-oauth-config\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.090771 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.090717    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-serving-cert\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.097244 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.097226    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfspn\" (UniqueName: \"kubernetes.io/projected/80b27150-4ddd-467c-acec-c4fb66953e48-kube-api-access-rfspn\") pod \"console-6c44494d47-7g9f6\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") " pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.179477 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.179453    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:41.333700 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:41.333675    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c44494d47-7g9f6"]
Jul 17 13:51:41.336740 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:51:41.336713    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80b27150_4ddd_467c_acec_c4fb66953e48.slice/crio-c4003de28fae4cf7e0c2ad62ddbc9a83c290fcde120d128474e48ba75acbf768 WatchSource:0}: Error finding container c4003de28fae4cf7e0c2ad62ddbc9a83c290fcde120d128474e48ba75acbf768: Status 404 returned error can't find the container with id c4003de28fae4cf7e0c2ad62ddbc9a83c290fcde120d128474e48ba75acbf768
Jul 17 13:51:42.097687 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:42.097648    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c44494d47-7g9f6" event={"ID":"80b27150-4ddd-467c-acec-c4fb66953e48","Type":"ContainerStarted","Data":"652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d"}
Jul 17 13:51:42.098117 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:42.097708    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c44494d47-7g9f6" event={"ID":"80b27150-4ddd-467c-acec-c4fb66953e48","Type":"ContainerStarted","Data":"c4003de28fae4cf7e0c2ad62ddbc9a83c290fcde120d128474e48ba75acbf768"}
Jul 17 13:51:42.119235 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:42.119184    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c44494d47-7g9f6" podStartSLOduration=2.119164459 podStartE2EDuration="2.119164459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:51:42.118941306 +0000 UTC m=+387.825388143" watchObservedRunningTime="2026-07-17 13:51:42.119164459 +0000 UTC m=+387.825611294"
Jul 17 13:51:43.042711 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:43.042673    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 13:51:43.043049 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:43.042986    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="alertmanager" containerID="cri-o://66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb" gracePeriod=120
Jul 17 13:51:43.043175 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:43.043050    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy-web" containerID="cri-o://2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65" gracePeriod=120
Jul 17 13:51:43.043175 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:43.043074    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="config-reloader" containerID="cri-o://2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1" gracePeriod=120
Jul 17 13:51:43.043175 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:43.043102    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="prom-label-proxy" containerID="cri-o://6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e" gracePeriod=120
Jul 17 13:51:43.043323 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:43.043099    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy" containerID="cri-o://723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895" gracePeriod=120
Jul 17 13:51:43.043323 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:43.043049    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy-metric" containerID="cri-o://2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7" gracePeriod=120
Jul 17 13:51:44.108744 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.108711    2561 generic.go:356] "Generic (PLEG): container finished" podID="1f947df8-6cde-4cb6-a32e-032521497584" containerID="6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e" exitCode=0
Jul 17 13:51:44.108744 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.108736    2561 generic.go:356] "Generic (PLEG): container finished" podID="1f947df8-6cde-4cb6-a32e-032521497584" containerID="2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7" exitCode=0
Jul 17 13:51:44.108744 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.108745    2561 generic.go:356] "Generic (PLEG): container finished" podID="1f947df8-6cde-4cb6-a32e-032521497584" containerID="723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895" exitCode=0
Jul 17 13:51:44.109176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.108754    2561 generic.go:356] "Generic (PLEG): container finished" podID="1f947df8-6cde-4cb6-a32e-032521497584" containerID="2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1" exitCode=0
Jul 17 13:51:44.109176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.108762    2561 generic.go:356] "Generic (PLEG): container finished" podID="1f947df8-6cde-4cb6-a32e-032521497584" containerID="66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb" exitCode=0
Jul 17 13:51:44.109176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.108779    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerDied","Data":"6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e"}
Jul 17 13:51:44.109176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.108863    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerDied","Data":"2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7"}
Jul 17 13:51:44.109176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.108888    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerDied","Data":"723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895"}
Jul 17 13:51:44.109176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.108911    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerDied","Data":"2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1"}
Jul 17 13:51:44.109176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.108933    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerDied","Data":"66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb"}
Jul 17 13:51:44.287694 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.287672    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:44.415755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.415686    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-main-db\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.415755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.415718    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.415755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.415754    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-web\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.416037 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.415775    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-metrics-client-ca\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.416037 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.415805    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-config-volume\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.416037 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.415821    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.416037 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.415861    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-trusted-ca-bundle\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.416037 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.415892    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-web-config\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.416289 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.416262    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:51:44.416343 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.416320    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-cluster-tls-config\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.416397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.416355    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-config-out\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.416397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.416360    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:51:44.416397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.416384    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cs5rc\" (UniqueName: \"kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-kube-api-access-cs5rc\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.417315 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.416699    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.417315 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.416730    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-tls-assets\") pod \"1f947df8-6cde-4cb6-a32e-032521497584\" (UID: \"1f947df8-6cde-4cb6-a32e-032521497584\") "
Jul 17 13:51:44.417315 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.417051    2561 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-metrics-client-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.417315 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.417070    2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.417315 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.417028    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 13:51:44.418991 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.418963    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:51:44.419580 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.419545    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:51:44.419953 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.419926    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-config-out" (OuterVolumeSpecName: "config-out") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 13:51:44.420156 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.420122    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:51:44.420273 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.420248    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:51:44.420429 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.420407    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-config-volume" (OuterVolumeSpecName: "config-volume") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:51:44.420610 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.420586    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:51:44.420773 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.420750    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-kube-api-access-cs5rc" (OuterVolumeSpecName: "kube-api-access-cs5rc") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "kube-api-access-cs5rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:51:44.424264 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.424159    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:51:44.431522 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.431502    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-web-config" (OuterVolumeSpecName: "web-config") pod "1f947df8-6cde-4cb6-a32e-032521497584" (UID: "1f947df8-6cde-4cb6-a32e-032521497584"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:51:44.517634 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517601    2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-web-config\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.517634 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517633    2561 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-cluster-tls-config\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.517634 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517642    2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-config-out\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.517788 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517651    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cs5rc\" (UniqueName: \"kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-kube-api-access-cs5rc\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.517788 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517660    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-main-tls\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.517788 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517669    2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f947df8-6cde-4cb6-a32e-032521497584-tls-assets\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.517788 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517677    2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f947df8-6cde-4cb6-a32e-032521497584-alertmanager-main-db\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.517788 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517685    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.517788 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517694    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.517788 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517702    2561 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-config-volume\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:44.517788 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:44.517710    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f947df8-6cde-4cb6-a32e-032521497584-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:51:45.115725 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.115651    2561 generic.go:356] "Generic (PLEG): container finished" podID="1f947df8-6cde-4cb6-a32e-032521497584" containerID="2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65" exitCode=0
Jul 17 13:51:45.116086 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.115737    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.116086 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.115733    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerDied","Data":"2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65"}
Jul 17 13:51:45.116086 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.115802    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f947df8-6cde-4cb6-a32e-032521497584","Type":"ContainerDied","Data":"544067c3108c3ff4396a0b25d565d9d78a3ca07a3fc15ba646aa6a061dc0bb73"}
Jul 17 13:51:45.116086 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.115834    2561 scope.go:117] "RemoveContainer" containerID="6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e"
Jul 17 13:51:45.123725 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.123708    2561 scope.go:117] "RemoveContainer" containerID="2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7"
Jul 17 13:51:45.133119 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.133102    2561 scope.go:117] "RemoveContainer" containerID="723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895"
Jul 17 13:51:45.137597 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.137574    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 13:51:45.140272 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.140256    2561 scope.go:117] "RemoveContainer" containerID="2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65"
Jul 17 13:51:45.147215 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.147200    2561 scope.go:117] "RemoveContainer" containerID="2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1"
Jul 17 13:51:45.148501 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.148448    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 13:51:45.154140 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.153973    2561 scope.go:117] "RemoveContainer" containerID="66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb"
Jul 17 13:51:45.160743 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.160728    2561 scope.go:117] "RemoveContainer" containerID="ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46"
Jul 17 13:51:45.167231 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.167217    2561 scope.go:117] "RemoveContainer" containerID="6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e"
Jul 17 13:51:45.167464 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:51:45.167445    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e\": container with ID starting with 6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e not found: ID does not exist" containerID="6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e"
Jul 17 13:51:45.167520 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.167471    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e"} err="failed to get container status \"6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e\": rpc error: code = NotFound desc = could not find container \"6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e\": container with ID starting with 6b2245410b3d56f8282c696ade7de3d7fe4c58ae877f08244cacf0a13c9f127e not found: ID does not exist"
Jul 17 13:51:45.167520 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.167487    2561 scope.go:117] "RemoveContainer" containerID="2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7"
Jul 17 13:51:45.167687 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:51:45.167672    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7\": container with ID starting with 2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7 not found: ID does not exist" containerID="2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7"
Jul 17 13:51:45.167724 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.167692    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7"} err="failed to get container status \"2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7\": rpc error: code = NotFound desc = could not find container \"2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7\": container with ID starting with 2c3543ea27fff22cecc1a9534c0bf72808835414b45579481576bb507d45cbf7 not found: ID does not exist"
Jul 17 13:51:45.167724 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.167705    2561 scope.go:117] "RemoveContainer" containerID="723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895"
Jul 17 13:51:45.167925 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:51:45.167908    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895\": container with ID starting with 723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895 not found: ID does not exist" containerID="723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895"
Jul 17 13:51:45.167970 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.167930    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895"} err="failed to get container status \"723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895\": rpc error: code = NotFound desc = could not find container \"723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895\": container with ID starting with 723151323bf1b52dcc725862ffce36930431a23801a44e4f2e0d2b3b2b52c895 not found: ID does not exist"
Jul 17 13:51:45.167970 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.167946    2561 scope.go:117] "RemoveContainer" containerID="2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65"
Jul 17 13:51:45.168187 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:51:45.168172    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65\": container with ID starting with 2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65 not found: ID does not exist" containerID="2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65"
Jul 17 13:51:45.168231 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.168190    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65"} err="failed to get container status \"2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65\": rpc error: code = NotFound desc = could not find container \"2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65\": container with ID starting with 2295a1ef2473d0e36dabf2f69f51f5d24c36ce50730943b4fae36d5c3ac75e65 not found: ID does not exist"
Jul 17 13:51:45.168231 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.168202    2561 scope.go:117] "RemoveContainer" containerID="2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1"
Jul 17 13:51:45.168424 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:51:45.168408    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1\": container with ID starting with 2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1 not found: ID does not exist" containerID="2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1"
Jul 17 13:51:45.168458 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.168428    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1"} err="failed to get container status \"2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1\": rpc error: code = NotFound desc = could not find container \"2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1\": container with ID starting with 2d6d8473654b16b8bfade96413d0449c64bcd1604bd1b1dcc3607e71205dcaa1 not found: ID does not exist"
Jul 17 13:51:45.168458 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.168440    2561 scope.go:117] "RemoveContainer" containerID="66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb"
Jul 17 13:51:45.168624 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:51:45.168610    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb\": container with ID starting with 66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb not found: ID does not exist" containerID="66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb"
Jul 17 13:51:45.168660 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.168628    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb"} err="failed to get container status \"66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb\": rpc error: code = NotFound desc = could not find container \"66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb\": container with ID starting with 66b055fa7030457fdf6b5494e1937da600d55b58521e4b13661b3fe85ef31beb not found: ID does not exist"
Jul 17 13:51:45.168660 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.168650    2561 scope.go:117] "RemoveContainer" containerID="ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46"
Jul 17 13:51:45.168828 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:51:45.168814    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46\": container with ID starting with ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46 not found: ID does not exist" containerID="ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46"
Jul 17 13:51:45.168863 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.168831    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46"} err="failed to get container status \"ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46\": rpc error: code = NotFound desc = could not find container \"ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46\": container with ID starting with ff8adc81ae076d6681b6ae304bc2ed1f41797d4e5f742424c79abec4e88c0d46 not found: ID does not exist"
Jul 17 13:51:45.182093 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182066    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 13:51:45.182354 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182342    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="prom-label-proxy"
Jul 17 13:51:45.182397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182356    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="prom-label-proxy"
Jul 17 13:51:45.182397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182364    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="init-config-reloader"
Jul 17 13:51:45.182397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182369    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="init-config-reloader"
Jul 17 13:51:45.182397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182375    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="alertmanager"
Jul 17 13:51:45.182397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182381    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="alertmanager"
Jul 17 13:51:45.182397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182388    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy"
Jul 17 13:51:45.182397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182393    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182402    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy-metric"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182408    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy-metric"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182417    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="config-reloader"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182423    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="config-reloader"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182432    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy-web"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182437    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy-web"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182485    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy-metric"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182492    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="prom-label-proxy"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182497    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy-web"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182504    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="config-reloader"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182509    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="kube-rbac-proxy"
Jul 17 13:51:45.182600 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.182514    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f947df8-6cde-4cb6-a32e-032521497584" containerName="alertmanager"
Jul 17 13:51:45.186513 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.186496    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.189727 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.189711    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 17 13:51:45.190261 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.190239    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 17 13:51:45.190342 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.190271    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 17 13:51:45.190342 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.190318    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 17 13:51:45.190342 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.190331    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 17 13:51:45.190494 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.190343    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jk946\""
Jul 17 13:51:45.190623 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.190607    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 17 13:51:45.190669 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.190610    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 17 13:51:45.190716 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.190613    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 17 13:51:45.195427 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.195411    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 17 13:51:45.217132 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.217110    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 13:51:45.323423 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323400    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e794b55-bc79-438d-863f-a922ce0f6bba-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323531 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323430    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323531 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323448    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e794b55-bc79-438d-863f-a922ce0f6bba-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323531 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323488    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323531 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323517    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e794b55-bc79-438d-863f-a922ce0f6bba-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323729 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323541    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e794b55-bc79-438d-863f-a922ce0f6bba-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323729 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323608    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-web-config\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323729 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323654    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323729 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323684    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323729 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323710    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7th7\" (UniqueName: \"kubernetes.io/projected/4e794b55-bc79-438d-863f-a922ce0f6bba-kube-api-access-b7th7\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323941 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323751    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323941 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323782    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e794b55-bc79-438d-863f-a922ce0f6bba-config-out\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.323941 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.323810    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425121 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425095    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425216 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425128    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e794b55-bc79-438d-863f-a922ce0f6bba-config-out\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425216 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425146    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425328 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425285    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e794b55-bc79-438d-863f-a922ce0f6bba-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425389 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425389 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425364    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e794b55-bc79-438d-863f-a922ce0f6bba-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425488 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425402    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425488 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425431    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e794b55-bc79-438d-863f-a922ce0f6bba-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425488 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425462    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e794b55-bc79-438d-863f-a922ce0f6bba-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425635 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425496    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-web-config\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425635 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425533    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425635 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425560    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425635 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425583    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7th7\" (UniqueName: \"kubernetes.io/projected/4e794b55-bc79-438d-863f-a922ce0f6bba-kube-api-access-b7th7\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.425806 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.425648    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e794b55-bc79-438d-863f-a922ce0f6bba-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.426510 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.426486    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e794b55-bc79-438d-863f-a922ce0f6bba-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.427411 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.427389    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e794b55-bc79-438d-863f-a922ce0f6bba-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.428275 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.428250    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e794b55-bc79-438d-863f-a922ce0f6bba-config-out\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.428566 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.428505    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.428665 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.428608    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.428665 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.428631    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e794b55-bc79-438d-863f-a922ce0f6bba-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.428867 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.428820    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-web-config\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.428867 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.428823    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.429089 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.429074    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.429845 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.429824    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.430397 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.430379    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e794b55-bc79-438d-863f-a922ce0f6bba-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.434602 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.434581    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7th7\" (UniqueName: \"kubernetes.io/projected/4e794b55-bc79-438d-863f-a922ce0f6bba-kube-api-access-b7th7\") pod \"alertmanager-main-0\" (UID: \"4e794b55-bc79-438d-863f-a922ce0f6bba\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.499501 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.499471    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 13:51:45.657749 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:45.657719    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 13:51:45.660709 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:51:45.660684    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e794b55_bc79_438d_863f_a922ce0f6bba.slice/crio-0ea73fc6f3ad5637c09bf71a77ebde94bbdc78dcd9dcd581d28d47a4d0cb47dd WatchSource:0}: Error finding container 0ea73fc6f3ad5637c09bf71a77ebde94bbdc78dcd9dcd581d28d47a4d0cb47dd: Status 404 returned error can't find the container with id 0ea73fc6f3ad5637c09bf71a77ebde94bbdc78dcd9dcd581d28d47a4d0cb47dd
Jul 17 13:51:46.123352 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:46.123276    2561 generic.go:356] "Generic (PLEG): container finished" podID="4e794b55-bc79-438d-863f-a922ce0f6bba" containerID="448242c7de736feac07047c7f96e58307505e3310dacc57ef19f07ffae559011" exitCode=0
Jul 17 13:51:46.123825 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:46.123365    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e794b55-bc79-438d-863f-a922ce0f6bba","Type":"ContainerDied","Data":"448242c7de736feac07047c7f96e58307505e3310dacc57ef19f07ffae559011"}
Jul 17 13:51:46.123825 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:46.123434    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e794b55-bc79-438d-863f-a922ce0f6bba","Type":"ContainerStarted","Data":"0ea73fc6f3ad5637c09bf71a77ebde94bbdc78dcd9dcd581d28d47a4d0cb47dd"}
Jul 17 13:51:46.772670 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:46.772640    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f947df8-6cde-4cb6-a32e-032521497584" path="/var/lib/kubelet/pods/1f947df8-6cde-4cb6-a32e-032521497584/volumes"
Jul 17 13:51:47.133266 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:47.133178    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e794b55-bc79-438d-863f-a922ce0f6bba","Type":"ContainerStarted","Data":"7419b3ab62334c2ab75121f4f83e34d0ec511f4f677b664144f242358cee1cb2"}
Jul 17 13:51:47.133266 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:47.133265    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e794b55-bc79-438d-863f-a922ce0f6bba","Type":"ContainerStarted","Data":"9a61d2edeaaa66f404afbd7ab8e7d0950c64a5e555e28ea088f7d5690026ebc6"}
Jul 17 13:51:47.133746 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:47.133305    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e794b55-bc79-438d-863f-a922ce0f6bba","Type":"ContainerStarted","Data":"a000b8fece0bc49adb3b0462e0580520add75207125a5829e59b671d98ead001"}
Jul 17 13:51:47.133746 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:47.133342    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e794b55-bc79-438d-863f-a922ce0f6bba","Type":"ContainerStarted","Data":"a3585fe42ca42dd06ceddbcc71913e1ab49b175628e8d4ac036807cae76ecd3e"}
Jul 17 13:51:47.133746 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:47.133379    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e794b55-bc79-438d-863f-a922ce0f6bba","Type":"ContainerStarted","Data":"67e34e83f62b94f851042056ead2d779b748a152b43cff4507189e0b78e8bc8d"}
Jul 17 13:51:47.133746 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:47.133414    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e794b55-bc79-438d-863f-a922ce0f6bba","Type":"ContainerStarted","Data":"75d8929e290b69eba258c47a2cc066e25853114ab601126a2b5233ff98e4712e"}
Jul 17 13:51:47.198236 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:47.198194    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.183154525 podStartE2EDuration="2.198180537s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.026012ms" isStatefulPod=false podCreationTimestamp="2026-07-17 13:51:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:51:47.193840383 +0000 UTC m=+392.900287216" watchObservedRunningTime="2026-07-17 13:51:47.198180537 +0000 UTC m=+392.904627370"
Jul 17 13:51:51.179723 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:51.179682    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:51.180234 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:51.179748    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:51.184595 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:51.184573    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:52.164107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:52.164079    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:51:52.214759 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:51:52.214732    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f75bf7855-kvffm"]
Jul 17 13:52:17.235017 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.234960    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f75bf7855-kvffm" podUID="62494c12-5cdf-4847-920b-0f125a4db466" containerName="console" containerID="cri-o://25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa" gracePeriod=15
Jul 17 13:52:17.481173 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.481152    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:52:17.673645 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.673622    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-oauth-serving-cert\") pod \"62494c12-5cdf-4847-920b-0f125a4db466\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") "
Jul 17 13:52:17.673803 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.673675    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-console-config\") pod \"62494c12-5cdf-4847-920b-0f125a4db466\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") "
Jul 17 13:52:17.673803 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.673716    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-trusted-ca-bundle\") pod \"62494c12-5cdf-4847-920b-0f125a4db466\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") "
Jul 17 13:52:17.673803 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.673776    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-oauth-config\") pod \"62494c12-5cdf-4847-920b-0f125a4db466\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") "
Jul 17 13:52:17.673803 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.673800    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-service-ca\") pod \"62494c12-5cdf-4847-920b-0f125a4db466\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") "
Jul 17 13:52:17.674020 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.673852    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gffs5\" (UniqueName: \"kubernetes.io/projected/62494c12-5cdf-4847-920b-0f125a4db466-kube-api-access-gffs5\") pod \"62494c12-5cdf-4847-920b-0f125a4db466\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") "
Jul 17 13:52:17.674020 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.673889    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-serving-cert\") pod \"62494c12-5cdf-4847-920b-0f125a4db466\" (UID: \"62494c12-5cdf-4847-920b-0f125a4db466\") "
Jul 17 13:52:17.674163 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.674133    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "62494c12-5cdf-4847-920b-0f125a4db466" (UID: "62494c12-5cdf-4847-920b-0f125a4db466"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:52:17.674224 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.674189    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-console-config" (OuterVolumeSpecName: "console-config") pod "62494c12-5cdf-4847-920b-0f125a4db466" (UID: "62494c12-5cdf-4847-920b-0f125a4db466"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:52:17.674224 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.674215    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-service-ca" (OuterVolumeSpecName: "service-ca") pod "62494c12-5cdf-4847-920b-0f125a4db466" (UID: "62494c12-5cdf-4847-920b-0f125a4db466"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:52:17.674341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.674235    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "62494c12-5cdf-4847-920b-0f125a4db466" (UID: "62494c12-5cdf-4847-920b-0f125a4db466"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:52:17.676130 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.676102    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62494c12-5cdf-4847-920b-0f125a4db466-kube-api-access-gffs5" (OuterVolumeSpecName: "kube-api-access-gffs5") pod "62494c12-5cdf-4847-920b-0f125a4db466" (UID: "62494c12-5cdf-4847-920b-0f125a4db466"). InnerVolumeSpecName "kube-api-access-gffs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:52:17.676246 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.676162    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "62494c12-5cdf-4847-920b-0f125a4db466" (UID: "62494c12-5cdf-4847-920b-0f125a4db466"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:52:17.676246 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.676185    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "62494c12-5cdf-4847-920b-0f125a4db466" (UID: "62494c12-5cdf-4847-920b-0f125a4db466"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:52:17.774862 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.774836    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-trusted-ca-bundle\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:52:17.774862 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.774858    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-oauth-config\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:52:17.775026 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.774869    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:52:17.775026 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.774879    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gffs5\" (UniqueName: \"kubernetes.io/projected/62494c12-5cdf-4847-920b-0f125a4db466-kube-api-access-gffs5\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:52:17.775026 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.774887    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62494c12-5cdf-4847-920b-0f125a4db466-console-serving-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:52:17.775026 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.774895    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-oauth-serving-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:52:17.775026 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:17.774902    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62494c12-5cdf-4847-920b-0f125a4db466-console-config\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:52:18.277941 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:18.277906    2561 generic.go:356] "Generic (PLEG): container finished" podID="62494c12-5cdf-4847-920b-0f125a4db466" containerID="25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa" exitCode=0
Jul 17 13:52:18.278329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:18.277949    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f75bf7855-kvffm"
Jul 17 13:52:18.278329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:18.277956    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f75bf7855-kvffm" event={"ID":"62494c12-5cdf-4847-920b-0f125a4db466","Type":"ContainerDied","Data":"25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa"}
Jul 17 13:52:18.278329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:18.277995    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f75bf7855-kvffm" event={"ID":"62494c12-5cdf-4847-920b-0f125a4db466","Type":"ContainerDied","Data":"e44f6093e0af9595de139ab8d0b7fa80ea863a6a5338c849f0132135ffb44937"}
Jul 17 13:52:18.278329 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:18.278031    2561 scope.go:117] "RemoveContainer" containerID="25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa"
Jul 17 13:52:18.292301 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:18.292283    2561 scope.go:117] "RemoveContainer" containerID="25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa"
Jul 17 13:52:18.292549 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:52:18.292532    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa\": container with ID starting with 25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa not found: ID does not exist" containerID="25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa"
Jul 17 13:52:18.292613 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:18.292554    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa"} err="failed to get container status \"25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa\": rpc error: code = NotFound desc = could not find container \"25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa\": container with ID starting with 25129ba56e3e2c7f75979f5259ebaa11f57e69eb2fd304e3086efe50caaec2aa not found: ID does not exist"
Jul 17 13:52:18.304039 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:18.303996    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f75bf7855-kvffm"]
Jul 17 13:52:18.310859 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:18.310841    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f75bf7855-kvffm"]
Jul 17 13:52:18.770442 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:52:18.770416    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62494c12-5cdf-4847-920b-0f125a4db466" path="/var/lib/kubelet/pods/62494c12-5cdf-4847-920b-0f125a4db466/volumes"
Jul 17 13:53:13.641675 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.641644    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-lh8wk"]
Jul 17 13:53:13.642135 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.641955    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62494c12-5cdf-4847-920b-0f125a4db466" containerName="console"
Jul 17 13:53:13.642135 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.641964    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="62494c12-5cdf-4847-920b-0f125a4db466" containerName="console"
Jul 17 13:53:13.642135 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.642031    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="62494c12-5cdf-4847-920b-0f125a4db466" containerName="console"
Jul 17 13:53:13.644900 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.644884    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk"
Jul 17 13:53:13.647832 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.647811    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 17 13:53:13.647832 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.647819    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 17 13:53:13.659580 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.659559    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-lh8wk"]
Jul 17 13:53:13.807781 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.807745    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dglwc\" (UniqueName: \"kubernetes.io/projected/9bd109d0-5314-4f33-a940-b87d0c6ac508-kube-api-access-dglwc\") pod \"cert-manager-webhook-6486ffd997-lh8wk\" (UID: \"9bd109d0-5314-4f33-a940-b87d0c6ac508\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk"
Jul 17 13:53:13.807781 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.807780    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9bd109d0-5314-4f33-a940-b87d0c6ac508-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-lh8wk\" (UID: \"9bd109d0-5314-4f33-a940-b87d0c6ac508\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk"
Jul 17 13:53:13.908886 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.908819    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dglwc\" (UniqueName: \"kubernetes.io/projected/9bd109d0-5314-4f33-a940-b87d0c6ac508-kube-api-access-dglwc\") pod \"cert-manager-webhook-6486ffd997-lh8wk\" (UID: \"9bd109d0-5314-4f33-a940-b87d0c6ac508\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk"
Jul 17 13:53:13.908886 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.908853    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9bd109d0-5314-4f33-a940-b87d0c6ac508-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-lh8wk\" (UID: \"9bd109d0-5314-4f33-a940-b87d0c6ac508\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk"
Jul 17 13:53:13.917852 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.917824    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9bd109d0-5314-4f33-a940-b87d0c6ac508-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-lh8wk\" (UID: \"9bd109d0-5314-4f33-a940-b87d0c6ac508\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk"
Jul 17 13:53:13.918044 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.918027    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dglwc\" (UniqueName: \"kubernetes.io/projected/9bd109d0-5314-4f33-a940-b87d0c6ac508-kube-api-access-dglwc\") pod \"cert-manager-webhook-6486ffd997-lh8wk\" (UID: \"9bd109d0-5314-4f33-a940-b87d0c6ac508\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk"
Jul 17 13:53:13.963443 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:13.963424    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk"
Jul 17 13:53:14.095359 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:14.095309    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-lh8wk"]
Jul 17 13:53:14.098317 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:53:14.098286    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bd109d0_5314_4f33_a940_b87d0c6ac508.slice/crio-fc7000625e7c4058babce96da222123730014f8ab87f37042c1415b9aac21fb7 WatchSource:0}: Error finding container fc7000625e7c4058babce96da222123730014f8ab87f37042c1415b9aac21fb7: Status 404 returned error can't find the container with id fc7000625e7c4058babce96da222123730014f8ab87f37042c1415b9aac21fb7
Jul 17 13:53:14.499607 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:14.499574    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk" event={"ID":"9bd109d0-5314-4f33-a940-b87d0c6ac508","Type":"ContainerStarted","Data":"fc7000625e7c4058babce96da222123730014f8ab87f37042c1415b9aac21fb7"}
Jul 17 13:53:17.516272 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:17.516238    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk" event={"ID":"9bd109d0-5314-4f33-a940-b87d0c6ac508","Type":"ContainerStarted","Data":"5382e901f9805b07242e84cd312bcd3351e58ebbc5715364ee01d3dd20cce0fb"}
Jul 17 13:53:17.516635 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:17.516299    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk"
Jul 17 13:53:17.535256 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:17.535199    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk" podStartSLOduration=1.281923412 podStartE2EDuration="4.535181354s" totalImagesPullingTime="3.253257942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:53:17.534413617 +0000 UTC m=+483.240860462" watchObservedRunningTime="2026-07-17 13:53:17.535181354 +0000 UTC m=+483.241628188"
Jul 17 13:53:23.522187 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:23.522153    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-lh8wk"
Jul 17 13:53:27.442617 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.442584    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"]
Jul 17 13:53:27.446138 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.446118    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"
Jul 17 13:53:27.448716 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.448696    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 17 13:53:27.448806 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.448778    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 17 13:53:27.449622 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.449606    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-fqnl4\""
Jul 17 13:53:27.457351 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.457332    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"]
Jul 17 13:53:27.620995 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.620970    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25d89ea4-20ad-4571-b01c-5a3d727cefc7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bsl84\" (UID: \"25d89ea4-20ad-4571-b01c-5a3d727cefc7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"
Jul 17 13:53:27.621160 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.621035    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnwrb\" (UniqueName: \"kubernetes.io/projected/25d89ea4-20ad-4571-b01c-5a3d727cefc7-kube-api-access-fnwrb\") pod \"openshift-lws-operator-fd8ccff4c-bsl84\" (UID: \"25d89ea4-20ad-4571-b01c-5a3d727cefc7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"
Jul 17 13:53:27.721815 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.721752    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25d89ea4-20ad-4571-b01c-5a3d727cefc7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bsl84\" (UID: \"25d89ea4-20ad-4571-b01c-5a3d727cefc7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"
Jul 17 13:53:27.721815 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.721807    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnwrb\" (UniqueName: \"kubernetes.io/projected/25d89ea4-20ad-4571-b01c-5a3d727cefc7-kube-api-access-fnwrb\") pod \"openshift-lws-operator-fd8ccff4c-bsl84\" (UID: \"25d89ea4-20ad-4571-b01c-5a3d727cefc7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"
Jul 17 13:53:27.722270 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.722250    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25d89ea4-20ad-4571-b01c-5a3d727cefc7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bsl84\" (UID: \"25d89ea4-20ad-4571-b01c-5a3d727cefc7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"
Jul 17 13:53:27.734548 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.732066    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnwrb\" (UniqueName: \"kubernetes.io/projected/25d89ea4-20ad-4571-b01c-5a3d727cefc7-kube-api-access-fnwrb\") pod \"openshift-lws-operator-fd8ccff4c-bsl84\" (UID: \"25d89ea4-20ad-4571-b01c-5a3d727cefc7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"
Jul 17 13:53:27.754698 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.754670    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"
Jul 17 13:53:27.882798 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:27.882731    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84"]
Jul 17 13:53:27.884979 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:53:27.884953    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25d89ea4_20ad_4571_b01c_5a3d727cefc7.slice/crio-585eedbac06255f5f3c4c3b57e065021baab0abc81bab17ab5218c9811bde4a5 WatchSource:0}: Error finding container 585eedbac06255f5f3c4c3b57e065021baab0abc81bab17ab5218c9811bde4a5: Status 404 returned error can't find the container with id 585eedbac06255f5f3c4c3b57e065021baab0abc81bab17ab5218c9811bde4a5
Jul 17 13:53:28.565440 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:28.565374    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84" event={"ID":"25d89ea4-20ad-4571-b01c-5a3d727cefc7","Type":"ContainerStarted","Data":"585eedbac06255f5f3c4c3b57e065021baab0abc81bab17ab5218c9811bde4a5"}
Jul 17 13:53:30.575993 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:30.575958    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84" event={"ID":"25d89ea4-20ad-4571-b01c-5a3d727cefc7","Type":"ContainerStarted","Data":"fd1441ce3c0890f4cb303ae8e4a6357eabe52d1f1e09576b721743c6f4f373c8"}
Jul 17 13:53:30.600671 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:30.600620    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bsl84" podStartSLOduration=1.32676391 podStartE2EDuration="3.600607539s" totalImagesPullingTime="2.273843629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:53:30.599321766 +0000 UTC m=+496.305768602" watchObservedRunningTime="2026-07-17 13:53:30.600607539 +0000 UTC m=+496.307054372"
Jul 17 13:53:49.021749 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.021713    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"]
Jul 17 13:53:49.025316 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.025295    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.028082 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.028062    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pxqwq\""
Jul 17 13:53:49.028338 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.028324    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 13:53:49.028590 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.028575    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 13:53:49.028739 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.028726    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 13:53:49.037827 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.037808    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 13:53:49.069770 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.069747    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"]
Jul 17 13:53:49.110762 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.110740    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4j4t\" (UniqueName: \"kubernetes.io/projected/4d74f61d-894d-4405-8f9d-577513bb5489-kube-api-access-d4j4t\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.110862 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.110772    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d74f61d-894d-4405-8f9d-577513bb5489-webhook-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.110862 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.110794    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4d74f61d-894d-4405-8f9d-577513bb5489-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.110862 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.110810    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4d74f61d-894d-4405-8f9d-577513bb5489-manifests\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.110971 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.110934    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d74f61d-894d-4405-8f9d-577513bb5489-tmp\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.211587 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.211557    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4j4t\" (UniqueName: \"kubernetes.io/projected/4d74f61d-894d-4405-8f9d-577513bb5489-kube-api-access-d4j4t\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.211587 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.211593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d74f61d-894d-4405-8f9d-577513bb5489-webhook-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.211794 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.211616    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4d74f61d-894d-4405-8f9d-577513bb5489-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.211794 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.211635    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4d74f61d-894d-4405-8f9d-577513bb5489-manifests\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.211794 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.211675    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d74f61d-894d-4405-8f9d-577513bb5489-tmp\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.212250 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.212217    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4d74f61d-894d-4405-8f9d-577513bb5489-manifests\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.214784 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.214756    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d74f61d-894d-4405-8f9d-577513bb5489-webhook-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.215232 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.215194    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4d74f61d-894d-4405-8f9d-577513bb5489-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.215500 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.215474    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d74f61d-894d-4405-8f9d-577513bb5489-tmp\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.227605 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.227584    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4j4t\" (UniqueName: \"kubernetes.io/projected/4d74f61d-894d-4405-8f9d-577513bb5489-kube-api-access-d4j4t\") pod \"opendatahub-operator-controller-manager-57f79678b4-wjnpf\" (UID: \"4d74f61d-894d-4405-8f9d-577513bb5489\") " pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.335738 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.335655    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:49.479139 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.479024    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"]
Jul 17 13:53:49.481753 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:53:49.481725    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d74f61d_894d_4405_8f9d_577513bb5489.slice/crio-079218dcfdae11924337c603b4b253a2ae850f9cb69d238f6cad4ab1e8f50902 WatchSource:0}: Error finding container 079218dcfdae11924337c603b4b253a2ae850f9cb69d238f6cad4ab1e8f50902: Status 404 returned error can't find the container with id 079218dcfdae11924337c603b4b253a2ae850f9cb69d238f6cad4ab1e8f50902
Jul 17 13:53:49.656749 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:49.656688    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf" event={"ID":"4d74f61d-894d-4405-8f9d-577513bb5489","Type":"ContainerStarted","Data":"079218dcfdae11924337c603b4b253a2ae850f9cb69d238f6cad4ab1e8f50902"}
Jul 17 13:53:52.674639 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:52.674609    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf" event={"ID":"4d74f61d-894d-4405-8f9d-577513bb5489","Type":"ContainerStarted","Data":"23b266f52507d1a19f60216286bf65e4ad0a18136e5581233b316fb881e733c8"}
Jul 17 13:53:53.680151 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:53.680115    2561 generic.go:356] "Generic (PLEG): container finished" podID="4d74f61d-894d-4405-8f9d-577513bb5489" containerID="23b266f52507d1a19f60216286bf65e4ad0a18136e5581233b316fb881e733c8" exitCode=0
Jul 17 13:53:53.680504 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:53.680193    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf" event={"ID":"4d74f61d-894d-4405-8f9d-577513bb5489","Type":"ContainerDied","Data":"23b266f52507d1a19f60216286bf65e4ad0a18136e5581233b316fb881e733c8"}
Jul 17 13:53:54.687596 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:54.687561    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf" event={"ID":"4d74f61d-894d-4405-8f9d-577513bb5489","Type":"ContainerStarted","Data":"6dd58e18b8efa65d0a1e227e0e79b90aad508d126049d8066820eee75bc3baaa"}
Jul 17 13:53:54.688071 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:54.687686    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:53:54.712683 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:53:54.712645    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf" podStartSLOduration=2.378707361 podStartE2EDuration="6.71263047s" totalImagesPullingTime="3.29474093s" totalInitContainerRuntime="1.039182179s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:53:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:53:54.71079906 +0000 UTC m=+520.417245914" watchObservedRunningTime="2026-07-17 13:53:54.71263047 +0000 UTC m=+520.419077304"
Jul 17 13:54:05.693829 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:05.693753    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57f79678b4-wjnpf"
Jul 17 13:54:08.917393 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:08.917363    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gvxmw"]
Jul 17 13:54:08.920988 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:08.920965    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:08.924318 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:08.924297    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sjn95\""
Jul 17 13:54:08.925530 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:08.925514    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 17 13:54:08.934836 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:08.934814    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gvxmw"]
Jul 17 13:54:09.083303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.083258    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9jcd\" (UniqueName: \"kubernetes.io/projected/bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6-kube-api-access-k9jcd\") pod \"odh-model-controller-7998c67dc4-gvxmw\" (UID: \"bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:09.083303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.083304    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6-cert\") pod \"odh-model-controller-7998c67dc4-gvxmw\" (UID: \"bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:09.184876 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.184785    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9jcd\" (UniqueName: \"kubernetes.io/projected/bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6-kube-api-access-k9jcd\") pod \"odh-model-controller-7998c67dc4-gvxmw\" (UID: \"bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:09.184876 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.184821    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6-cert\") pod \"odh-model-controller-7998c67dc4-gvxmw\" (UID: \"bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:09.185147 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:54:09.184964    2561 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 17 13:54:09.185147 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:54:09.185054    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6-cert podName:bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6 nodeName:}" failed. No retries permitted until 2026-07-17 13:54:09.685033327 +0000 UTC m=+535.391480160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6-cert") pod "odh-model-controller-7998c67dc4-gvxmw" (UID: "bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6") : secret "odh-model-controller-webhook-cert" not found
Jul 17 13:54:09.194484 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.194458    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9jcd\" (UniqueName: \"kubernetes.io/projected/bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6-kube-api-access-k9jcd\") pod \"odh-model-controller-7998c67dc4-gvxmw\" (UID: \"bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:09.627922 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.627841    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lhvbm"]
Jul 17 13:54:09.631559 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.631537    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:09.634124 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.634103    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-fjjfs\""
Jul 17 13:54:09.634893 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.634872    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 17 13:54:09.643130 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.643106    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lhvbm"]
Jul 17 13:54:09.690388 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.690362    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6-cert\") pod \"odh-model-controller-7998c67dc4-gvxmw\" (UID: \"bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:09.693052 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.693032    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6-cert\") pod \"odh-model-controller-7998c67dc4-gvxmw\" (UID: \"bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:09.791576 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.791543    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8276bb-9bf9-469e-b299-c4eb405e4b1b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lhvbm\" (UID: \"6b8276bb-9bf9-469e-b299-c4eb405e4b1b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:09.791718 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.791595    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzfvw\" (UniqueName: \"kubernetes.io/projected/6b8276bb-9bf9-469e-b299-c4eb405e4b1b-kube-api-access-jzfvw\") pod \"model-serving-api-54d89bd5fc-lhvbm\" (UID: \"6b8276bb-9bf9-469e-b299-c4eb405e4b1b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:09.831467 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.831440    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:09.894728 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.893514    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8276bb-9bf9-469e-b299-c4eb405e4b1b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lhvbm\" (UID: \"6b8276bb-9bf9-469e-b299-c4eb405e4b1b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:09.894728 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.893574    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzfvw\" (UniqueName: \"kubernetes.io/projected/6b8276bb-9bf9-469e-b299-c4eb405e4b1b-kube-api-access-jzfvw\") pod \"model-serving-api-54d89bd5fc-lhvbm\" (UID: \"6b8276bb-9bf9-469e-b299-c4eb405e4b1b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:09.894728 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:54:09.894084    2561 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 17 13:54:09.894728 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:54:09.894170    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8276bb-9bf9-469e-b299-c4eb405e4b1b-tls-certs podName:6b8276bb-9bf9-469e-b299-c4eb405e4b1b nodeName:}" failed. No retries permitted until 2026-07-17 13:54:10.394146895 +0000 UTC m=+536.100593730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6b8276bb-9bf9-469e-b299-c4eb405e4b1b-tls-certs") pod "model-serving-api-54d89bd5fc-lhvbm" (UID: "6b8276bb-9bf9-469e-b299-c4eb405e4b1b") : secret "model-serving-api-tls" not found
Jul 17 13:54:09.907177 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.907115    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzfvw\" (UniqueName: \"kubernetes.io/projected/6b8276bb-9bf9-469e-b299-c4eb405e4b1b-kube-api-access-jzfvw\") pod \"model-serving-api-54d89bd5fc-lhvbm\" (UID: \"6b8276bb-9bf9-469e-b299-c4eb405e4b1b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:09.985211 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:09.985100    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gvxmw"]
Jul 17 13:54:09.988032 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:54:09.987986    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe2fe4c_c507_4b75_9e2a_d13ebf1600f6.slice/crio-91795e1503f27f9b3f1c99949d52544359de4e624394d636a905e2ee4c8ea237 WatchSource:0}: Error finding container 91795e1503f27f9b3f1c99949d52544359de4e624394d636a905e2ee4c8ea237: Status 404 returned error can't find the container with id 91795e1503f27f9b3f1c99949d52544359de4e624394d636a905e2ee4c8ea237
Jul 17 13:54:10.398879 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:10.398843    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8276bb-9bf9-469e-b299-c4eb405e4b1b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lhvbm\" (UID: \"6b8276bb-9bf9-469e-b299-c4eb405e4b1b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:10.401693 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:10.401663    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8276bb-9bf9-469e-b299-c4eb405e4b1b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lhvbm\" (UID: \"6b8276bb-9bf9-469e-b299-c4eb405e4b1b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:10.548609 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:10.548563    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:10.685960 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:10.685926    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lhvbm"]
Jul 17 13:54:10.688943 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:54:10.688918    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8276bb_9bf9_469e_b299_c4eb405e4b1b.slice/crio-ed689b66073f42c98ae541c73bcfeda6b85e2bc327d4f1840504477cd5949149 WatchSource:0}: Error finding container ed689b66073f42c98ae541c73bcfeda6b85e2bc327d4f1840504477cd5949149: Status 404 returned error can't find the container with id ed689b66073f42c98ae541c73bcfeda6b85e2bc327d4f1840504477cd5949149
Jul 17 13:54:10.763416 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:10.763340    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm" event={"ID":"6b8276bb-9bf9-469e-b299-c4eb405e4b1b","Type":"ContainerStarted","Data":"ed689b66073f42c98ae541c73bcfeda6b85e2bc327d4f1840504477cd5949149"}
Jul 17 13:54:10.773689 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:10.773653    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw" event={"ID":"bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6","Type":"ContainerStarted","Data":"91795e1503f27f9b3f1c99949d52544359de4e624394d636a905e2ee4c8ea237"}
Jul 17 13:54:11.759599 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.759569    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"]
Jul 17 13:54:11.769732 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.769704    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:11.774778 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.773192    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 17 13:54:11.774778 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.773225    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 13:54:11.774778 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.773456    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-d5c64\""
Jul 17 13:54:11.774778 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.774711    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"]
Jul 17 13:54:11.811538 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.811489    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28jsp\" (UniqueName: \"kubernetes.io/projected/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-kube-api-access-28jsp\") pod \"kube-auth-proxy-66c48b9c48-wv9s2\" (UID: \"26f08e15-82c5-4ad5-aafe-8f0a602f98a0\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:11.811710 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.811554    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-tmp\") pod \"kube-auth-proxy-66c48b9c48-wv9s2\" (UID: \"26f08e15-82c5-4ad5-aafe-8f0a602f98a0\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:11.811710 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.811593    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-tls-certs\") pod \"kube-auth-proxy-66c48b9c48-wv9s2\" (UID: \"26f08e15-82c5-4ad5-aafe-8f0a602f98a0\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:11.912570 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.912535    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-tmp\") pod \"kube-auth-proxy-66c48b9c48-wv9s2\" (UID: \"26f08e15-82c5-4ad5-aafe-8f0a602f98a0\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:11.912749 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.912596    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-tls-certs\") pod \"kube-auth-proxy-66c48b9c48-wv9s2\" (UID: \"26f08e15-82c5-4ad5-aafe-8f0a602f98a0\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:11.912749 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.912689    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28jsp\" (UniqueName: \"kubernetes.io/projected/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-kube-api-access-28jsp\") pod \"kube-auth-proxy-66c48b9c48-wv9s2\" (UID: \"26f08e15-82c5-4ad5-aafe-8f0a602f98a0\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:11.912872 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:54:11.912786    2561 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 17 13:54:11.912872 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:54:11.912869    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-tls-certs podName:26f08e15-82c5-4ad5-aafe-8f0a602f98a0 nodeName:}" failed. No retries permitted until 2026-07-17 13:54:12.412846254 +0000 UTC m=+538.119293074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-tls-certs") pod "kube-auth-proxy-66c48b9c48-wv9s2" (UID: "26f08e15-82c5-4ad5-aafe-8f0a602f98a0") : secret "kube-auth-proxy-tls" not found
Jul 17 13:54:11.915836 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.915809    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-tmp\") pod \"kube-auth-proxy-66c48b9c48-wv9s2\" (UID: \"26f08e15-82c5-4ad5-aafe-8f0a602f98a0\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:11.922228 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:11.922193    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28jsp\" (UniqueName: \"kubernetes.io/projected/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-kube-api-access-28jsp\") pod \"kube-auth-proxy-66c48b9c48-wv9s2\" (UID: \"26f08e15-82c5-4ad5-aafe-8f0a602f98a0\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:12.417935 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.417905    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-tls-certs\") pod \"kube-auth-proxy-66c48b9c48-wv9s2\" (UID: \"26f08e15-82c5-4ad5-aafe-8f0a602f98a0\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:12.420521 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.420489    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26f08e15-82c5-4ad5-aafe-8f0a602f98a0-tls-certs\") pod \"kube-auth-proxy-66c48b9c48-wv9s2\" (UID: \"26f08e15-82c5-4ad5-aafe-8f0a602f98a0\") " pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:12.684795 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.684710    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"
Jul 17 13:54:12.891109 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.891076    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"]
Jul 17 13:54:12.894915 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.894887    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:12.897749 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.897711    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 17 13:54:12.897749 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.897718    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 17 13:54:12.898120 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.897756    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 13:54:12.898120 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.897810    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rl9k4\""
Jul 17 13:54:12.898120 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.897718    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 17 13:54:12.905971 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.905949    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"]
Jul 17 13:54:12.922661 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.922630    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:12.922796 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.922691    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:12.922796 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.922763    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-token\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:12.922909 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.922806    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6dsk\" (UniqueName: \"kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-kube-api-access-k6dsk\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:12.922909 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.922868    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:12.923036 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.922909    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-cacerts\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:12.923036 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:12.922939    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4745e924-9fb9-4358-9acf-3dc1db98b63e-local-certs\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.024219 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.024089    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4745e924-9fb9-4358-9acf-3dc1db98b63e-local-certs\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.024383 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.024248    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.024383 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.024332    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.024383 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.024366    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-token\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.024546 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.024397    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6dsk\" (UniqueName: \"kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-kube-api-access-k6dsk\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.024546 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.024481    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.024546 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.024511    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-cacerts\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.025726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.025697    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.027085 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.027048    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4745e924-9fb9-4358-9acf-3dc1db98b63e-local-certs\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.027212 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.027046    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.027531 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.027483    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.033938 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.033914    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6dsk\" (UniqueName: \"kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-kube-api-access-k6dsk\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.034316 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.034295    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-token\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.040569 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.040546    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-cacerts\") pod \"istiod-openshift-gateway-7574459656-bbdfs\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.207898 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.207829    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:13.601583 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.601558    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2"]
Jul 17 13:54:13.603134 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:54:13.603110    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f08e15_82c5_4ad5_aafe_8f0a602f98a0.slice/crio-aa09901ed44a490e300f2675cc58f4538bca29bfd37673116ecef330d39493d8 WatchSource:0}: Error finding container aa09901ed44a490e300f2675cc58f4538bca29bfd37673116ecef330d39493d8: Status 404 returned error can't find the container with id aa09901ed44a490e300f2675cc58f4538bca29bfd37673116ecef330d39493d8
Jul 17 13:54:13.619835 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.619812    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"]
Jul 17 13:54:13.621649 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:54:13.621622    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4745e924_9fb9_4358_9acf_3dc1db98b63e.slice/crio-af37e4ad515b4b1cd65bcc016c3c3aa8c6a3cf22db64f42358b5ca03000b00a1 WatchSource:0}: Error finding container af37e4ad515b4b1cd65bcc016c3c3aa8c6a3cf22db64f42358b5ca03000b00a1: Status 404 returned error can't find the container with id af37e4ad515b4b1cd65bcc016c3c3aa8c6a3cf22db64f42358b5ca03000b00a1
Jul 17 13:54:13.785627 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.785540    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm" event={"ID":"6b8276bb-9bf9-469e-b299-c4eb405e4b1b","Type":"ContainerStarted","Data":"2705824c590ccddb2540726a0ae0d8484ddec72d36c9a88423eef490bfa342b9"}
Jul 17 13:54:13.785795 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.785641    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:13.787478 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.787449    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs" event={"ID":"4745e924-9fb9-4358-9acf-3dc1db98b63e","Type":"ContainerStarted","Data":"af37e4ad515b4b1cd65bcc016c3c3aa8c6a3cf22db64f42358b5ca03000b00a1"}
Jul 17 13:54:13.789320 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.789292    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw" event={"ID":"bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6","Type":"ContainerStarted","Data":"307b2019d88db1316f68439b9c5570ccb81b2a706a9f5764a8963a3ba5300b5f"}
Jul 17 13:54:13.789433 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.789421    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:13.790844 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.790822    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2" event={"ID":"26f08e15-82c5-4ad5-aafe-8f0a602f98a0","Type":"ContainerStarted","Data":"aa09901ed44a490e300f2675cc58f4538bca29bfd37673116ecef330d39493d8"}
Jul 17 13:54:13.807381 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.807333    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm" podStartSLOduration=2.05833988 podStartE2EDuration="4.807318114s" totalImagesPullingTime="2.748978234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:54:13.805578979 +0000 UTC m=+539.512025833" watchObservedRunningTime="2026-07-17 13:54:13.807318114 +0000 UTC m=+539.513764938"
Jul 17 13:54:13.824777 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:13.824731    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw" podStartSLOduration=2.376956216 podStartE2EDuration="5.824716708s" totalImagesPullingTime="3.447760492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:54:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:54:13.822760894 +0000 UTC m=+539.529207728" watchObservedRunningTime="2026-07-17 13:54:13.824716708 +0000 UTC m=+539.531163585"
Jul 17 13:54:16.968345 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:16.968301    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:54:16.968620 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:16.968390    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:54:16.975633 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:16.975616    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 13:54:17.817026 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:17.816969    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs" event={"ID":"4745e924-9fb9-4358-9acf-3dc1db98b63e","Type":"ContainerStarted","Data":"f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca"}
Jul 17 13:54:17.817201 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:17.817175    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:17.818646 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:17.818625    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2" event={"ID":"26f08e15-82c5-4ad5-aafe-8f0a602f98a0","Type":"ContainerStarted","Data":"8c6cba57262d334a2080551973dc30eec8467337e96fd2b567b94457100c3990"}
Jul 17 13:54:17.853825 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:17.853772    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs" podStartSLOduration=2.509997017 podStartE2EDuration="5.853756615s" totalImagesPullingTime="3.343759598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:54:17.852663602 +0000 UTC m=+543.559110441" watchObservedRunningTime="2026-07-17 13:54:17.853756615 +0000 UTC m=+543.560203426"
Jul 17 13:54:17.898616 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:17.898573    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66c48b9c48-wv9s2" podStartSLOduration=3.531195743 podStartE2EDuration="6.898560315s" totalImagesPullingTime="3.367364572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:54:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:54:17.895569292 +0000 UTC m=+543.602016144" watchObservedRunningTime="2026-07-17 13:54:17.898560315 +0000 UTC m=+543.605007149"
Jul 17 13:54:18.824504 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:18.824470    2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-bbdfs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 13:54:18.824931 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:18.824528    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs" podUID="4745e924-9fb9-4358-9acf-3dc1db98b63e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 13:54:19.076537 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.076467    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"]
Jul 17 13:54:19.079871 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.079856    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.082438 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.082410    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-sd4xk\""
Jul 17 13:54:19.098507 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.098484    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"]
Jul 17 13:54:19.187313 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.187281    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nkmg\" (UniqueName: \"kubernetes.io/projected/b60bfbcc-51ce-4006-b9de-a625cba35cb2-kube-api-access-4nkmg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.187510 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.187478    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.187663 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.187539    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.187663 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.187656    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.187787 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.187752    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.187869 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.187845    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.187930 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.187902    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.187983 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.187934    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.187983 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.187968    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.288826 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.288794    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.288826 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.288826    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.288851    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.288873    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.288903    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.288949    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nkmg\" (UniqueName: \"kubernetes.io/projected/b60bfbcc-51ce-4006-b9de-a625cba35cb2-kube-api-access-4nkmg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.288985    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.289046    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289364 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.289109    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289364 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.289271    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289430 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.289417    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.289722    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.289755 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.289730    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.290112 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.289941    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.291358 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.291331    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.291609 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.291593    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.303538 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.303513    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b60bfbcc-51ce-4006-b9de-a625cba35cb2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.310956 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.310937    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nkmg\" (UniqueName: \"kubernetes.io/projected/b60bfbcc-51ce-4006-b9de-a625cba35cb2-kube-api-access-4nkmg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ftdhm\" (UID: \"b60bfbcc-51ce-4006-b9de-a625cba35cb2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.391121 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.391025    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:19.549270 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.549244    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"]
Jul 17 13:54:19.550386 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:54:19.550364    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60bfbcc_51ce_4006_b9de_a625cba35cb2.slice/crio-24be8b6168f167f53cbdc22ece873db50c466a2a5fb248ddff753a11ac3335cd WatchSource:0}: Error finding container 24be8b6168f167f53cbdc22ece873db50c466a2a5fb248ddff753a11ac3335cd: Status 404 returned error can't find the container with id 24be8b6168f167f53cbdc22ece873db50c466a2a5fb248ddff753a11ac3335cd
Jul 17 13:54:19.828819 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:19.828778    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm" event={"ID":"b60bfbcc-51ce-4006-b9de-a625cba35cb2","Type":"ContainerStarted","Data":"24be8b6168f167f53cbdc22ece873db50c466a2a5fb248ddff753a11ac3335cd"}
Jul 17 13:54:21.519933 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.519897    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"]
Jul 17 13:54:21.524671 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.524641    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:54:21.526260 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.526235    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"]
Jul 17 13:54:21.529782 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.529733    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"
Jul 17 13:54:21.538750 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.538729    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 17 13:54:21.538847 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.538746    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-z9lzb\""
Jul 17 13:54:21.539129 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.539111    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dqncq\""
Jul 17 13:54:21.539904 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.539888    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 17 13:54:21.545938 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.545918    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"]
Jul 17 13:54:21.578333 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.578310    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"]
Jul 17 13:54:21.616896 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.616864    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a500635e-b087-4d94-888b-bada40827251-cert\") pod \"kserve-controller-manager-d6f4b65b8-hcggw\" (UID: \"a500635e-b087-4d94-888b-bada40827251\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:54:21.617069 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.616905    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crntl\" (UniqueName: \"kubernetes.io/projected/a500635e-b087-4d94-888b-bada40827251-kube-api-access-crntl\") pod \"kserve-controller-manager-d6f4b65b8-hcggw\" (UID: \"a500635e-b087-4d94-888b-bada40827251\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:54:21.617069 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.616944    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a275428-cc40-4134-9ed6-0f48a8c9d67c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-22dhv\" (UID: \"0a275428-cc40-4134-9ed6-0f48a8c9d67c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"
Jul 17 13:54:21.617069 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.616977    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44znf\" (UniqueName: \"kubernetes.io/projected/0a275428-cc40-4134-9ed6-0f48a8c9d67c-kube-api-access-44znf\") pod \"llmisvc-controller-manager-777c6bd7d5-22dhv\" (UID: \"0a275428-cc40-4134-9ed6-0f48a8c9d67c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"
Jul 17 13:54:21.718160 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.718128    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a500635e-b087-4d94-888b-bada40827251-cert\") pod \"kserve-controller-manager-d6f4b65b8-hcggw\" (UID: \"a500635e-b087-4d94-888b-bada40827251\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:54:21.718160 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.718164    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crntl\" (UniqueName: \"kubernetes.io/projected/a500635e-b087-4d94-888b-bada40827251-kube-api-access-crntl\") pod \"kserve-controller-manager-d6f4b65b8-hcggw\" (UID: \"a500635e-b087-4d94-888b-bada40827251\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:54:21.718396 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.718190    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a275428-cc40-4134-9ed6-0f48a8c9d67c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-22dhv\" (UID: \"0a275428-cc40-4134-9ed6-0f48a8c9d67c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"
Jul 17 13:54:21.718396 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.718216    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44znf\" (UniqueName: \"kubernetes.io/projected/0a275428-cc40-4134-9ed6-0f48a8c9d67c-kube-api-access-44znf\") pod \"llmisvc-controller-manager-777c6bd7d5-22dhv\" (UID: \"0a275428-cc40-4134-9ed6-0f48a8c9d67c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"
Jul 17 13:54:21.718396 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:54:21.718299    2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 17 13:54:21.718396 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:54:21.718375    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a500635e-b087-4d94-888b-bada40827251-cert podName:a500635e-b087-4d94-888b-bada40827251 nodeName:}" failed. No retries permitted until 2026-07-17 13:54:22.218352786 +0000 UTC m=+547.924799603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a500635e-b087-4d94-888b-bada40827251-cert") pod "kserve-controller-manager-d6f4b65b8-hcggw" (UID: "a500635e-b087-4d94-888b-bada40827251") : secret "kserve-webhook-server-cert" not found
Jul 17 13:54:21.720925 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.720895    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a275428-cc40-4134-9ed6-0f48a8c9d67c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-22dhv\" (UID: \"0a275428-cc40-4134-9ed6-0f48a8c9d67c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"
Jul 17 13:54:21.730758 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.730734    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crntl\" (UniqueName: \"kubernetes.io/projected/a500635e-b087-4d94-888b-bada40827251-kube-api-access-crntl\") pod \"kserve-controller-manager-d6f4b65b8-hcggw\" (UID: \"a500635e-b087-4d94-888b-bada40827251\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:54:21.730971 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.730950    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44znf\" (UniqueName: \"kubernetes.io/projected/0a275428-cc40-4134-9ed6-0f48a8c9d67c-kube-api-access-44znf\") pod \"llmisvc-controller-manager-777c6bd7d5-22dhv\" (UID: \"0a275428-cc40-4134-9ed6-0f48a8c9d67c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"
Jul 17 13:54:21.824102 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.824034    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:21.844464 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:21.844431    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"
Jul 17 13:54:22.014885 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.014856    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"]
Jul 17 13:54:22.016990 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:54:22.016961    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0a275428_cc40_4134_9ed6_0f48a8c9d67c.slice/crio-e50fd4fb68f87a2a8c05cf57517bc19ee3fca5600c451f6a6869bb01047ba418 WatchSource:0}: Error finding container e50fd4fb68f87a2a8c05cf57517bc19ee3fca5600c451f6a6869bb01047ba418: Status 404 returned error can't find the container with id e50fd4fb68f87a2a8c05cf57517bc19ee3fca5600c451f6a6869bb01047ba418
Jul 17 13:54:22.074729 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.074637    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"]
Jul 17 13:54:22.074894 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.074860    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs" podUID="4745e924-9fb9-4358-9acf-3dc1db98b63e" containerName="discovery" containerID="cri-o://f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca" gracePeriod=30
Jul 17 13:54:22.223741 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.223702    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a500635e-b087-4d94-888b-bada40827251-cert\") pod \"kserve-controller-manager-d6f4b65b8-hcggw\" (UID: \"a500635e-b087-4d94-888b-bada40827251\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:54:22.226834 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.226807    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a500635e-b087-4d94-888b-bada40827251-cert\") pod \"kserve-controller-manager-d6f4b65b8-hcggw\" (UID: \"a500635e-b087-4d94-888b-bada40827251\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:54:22.343616 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.343592    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:22.426118 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.426085    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-token\") pod \"4745e924-9fb9-4358-9acf-3dc1db98b63e\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") "
Jul 17 13:54:22.426301 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.426151    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-ca-configmap\") pod \"4745e924-9fb9-4358-9acf-3dc1db98b63e\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") "
Jul 17 13:54:22.426301 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.426197    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4745e924-9fb9-4358-9acf-3dc1db98b63e-local-certs\") pod \"4745e924-9fb9-4358-9acf-3dc1db98b63e\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") "
Jul 17 13:54:22.426301 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.426225    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6dsk\" (UniqueName: \"kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-kube-api-access-k6dsk\") pod \"4745e924-9fb9-4358-9acf-3dc1db98b63e\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") "
Jul 17 13:54:22.426301 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.426281    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-dns-cert\") pod \"4745e924-9fb9-4358-9acf-3dc1db98b63e\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") "
Jul 17 13:54:22.426500 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.426313    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-cacerts\") pod \"4745e924-9fb9-4358-9acf-3dc1db98b63e\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") "
Jul 17 13:54:22.426500 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.426374    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-kubeconfig\") pod \"4745e924-9fb9-4358-9acf-3dc1db98b63e\" (UID: \"4745e924-9fb9-4358-9acf-3dc1db98b63e\") "
Jul 17 13:54:22.426594 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.426567    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "4745e924-9fb9-4358-9acf-3dc1db98b63e" (UID: "4745e924-9fb9-4358-9acf-3dc1db98b63e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:54:22.426757 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.426737    2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-ca-configmap\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:54:22.429090 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.429049    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "4745e924-9fb9-4358-9acf-3dc1db98b63e" (UID: "4745e924-9fb9-4358-9acf-3dc1db98b63e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:54:22.429199 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.429099    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "4745e924-9fb9-4358-9acf-3dc1db98b63e" (UID: "4745e924-9fb9-4358-9acf-3dc1db98b63e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:54:22.429199 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.429130    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-kube-api-access-k6dsk" (OuterVolumeSpecName: "kube-api-access-k6dsk") pod "4745e924-9fb9-4358-9acf-3dc1db98b63e" (UID: "4745e924-9fb9-4358-9acf-3dc1db98b63e"). InnerVolumeSpecName "kube-api-access-k6dsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:54:22.429299 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.429205    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-token" (OuterVolumeSpecName: "istio-token") pod "4745e924-9fb9-4358-9acf-3dc1db98b63e" (UID: "4745e924-9fb9-4358-9acf-3dc1db98b63e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:54:22.429423 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.429400    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4745e924-9fb9-4358-9acf-3dc1db98b63e-local-certs" (OuterVolumeSpecName: "local-certs") pod "4745e924-9fb9-4358-9acf-3dc1db98b63e" (UID: "4745e924-9fb9-4358-9acf-3dc1db98b63e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 13:54:22.429457 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.429404    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-cacerts" (OuterVolumeSpecName: "cacerts") pod "4745e924-9fb9-4358-9acf-3dc1db98b63e" (UID: "4745e924-9fb9-4358-9acf-3dc1db98b63e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:54:22.438299 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.438279    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:54:22.527759 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.527729    2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-csr-dns-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:54:22.528193 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.527763    2561 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-cacerts\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:54:22.528193 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.527780    2561 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-kubeconfig\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:54:22.528193 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.527793    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:54:22.528193 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.527806    2561 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4745e924-9fb9-4358-9acf-3dc1db98b63e-local-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:54:22.528193 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.527819    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6dsk\" (UniqueName: \"kubernetes.io/projected/4745e924-9fb9-4358-9acf-3dc1db98b63e-kube-api-access-k6dsk\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:54:22.587031 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.586989    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"]
Jul 17 13:54:22.589219 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:54:22.589189    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda500635e_b087_4d94_888b_bada40827251.slice/crio-42567c7010c68707b5500fa9a7895205029ac24247718d7877a87984563ec5d3 WatchSource:0}: Error finding container 42567c7010c68707b5500fa9a7895205029ac24247718d7877a87984563ec5d3: Status 404 returned error can't find the container with id 42567c7010c68707b5500fa9a7895205029ac24247718d7877a87984563ec5d3
Jul 17 13:54:22.846690 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.846614    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw" event={"ID":"a500635e-b087-4d94-888b-bada40827251","Type":"ContainerStarted","Data":"42567c7010c68707b5500fa9a7895205029ac24247718d7877a87984563ec5d3"}
Jul 17 13:54:22.847796 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.847771    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv" event={"ID":"0a275428-cc40-4134-9ed6-0f48a8c9d67c","Type":"ContainerStarted","Data":"e50fd4fb68f87a2a8c05cf57517bc19ee3fca5600c451f6a6869bb01047ba418"}
Jul 17 13:54:22.850174 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.849069    2561 generic.go:356] "Generic (PLEG): container finished" podID="4745e924-9fb9-4358-9acf-3dc1db98b63e" containerID="f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca" exitCode=0
Jul 17 13:54:22.850174 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.849118    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs" event={"ID":"4745e924-9fb9-4358-9acf-3dc1db98b63e","Type":"ContainerDied","Data":"f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca"}
Jul 17 13:54:22.850174 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.849150    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs" event={"ID":"4745e924-9fb9-4358-9acf-3dc1db98b63e","Type":"ContainerDied","Data":"af37e4ad515b4b1cd65bcc016c3c3aa8c6a3cf22db64f42358b5ca03000b00a1"}
Jul 17 13:54:22.850174 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.849176    2561 scope.go:117] "RemoveContainer" containerID="f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca"
Jul 17 13:54:22.850174 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.849303    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"
Jul 17 13:54:22.875709 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.875678    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"]
Jul 17 13:54:22.878723 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.878688    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-bbdfs"]
Jul 17 13:54:22.934208 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.934182    2561 scope.go:117] "RemoveContainer" containerID="f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca"
Jul 17 13:54:22.934526 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:54:22.934507    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca\": container with ID starting with f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca not found: ID does not exist" containerID="f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca"
Jul 17 13:54:22.934601 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:22.934539    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca"} err="failed to get container status \"f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca\": rpc error: code = NotFound desc = could not find container \"f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca\": container with ID starting with f3810420863692dc608bac96c2b2975020436c5cca6365749b473c412f4432ca not found: ID does not exist"
Jul 17 13:54:24.773137 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:24.773102    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4745e924-9fb9-4358-9acf-3dc1db98b63e" path="/var/lib/kubelet/pods/4745e924-9fb9-4358-9acf-3dc1db98b63e/volumes"
Jul 17 13:54:24.802999 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:24.802454    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-lhvbm"
Jul 17 13:54:24.821517 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:24.821497    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-gvxmw"
Jul 17 13:54:26.793945 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:26.793911    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:54:26.794382 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:26.793988    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:54:26.794382 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:26.794046    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:54:26.878325 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:26.878290    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm" event={"ID":"b60bfbcc-51ce-4006-b9de-a625cba35cb2","Type":"ContainerStarted","Data":"578d6384c7301b0e34eb63ac3b4eab9bb83f06765d3c633782da04dc3ec41adb"}
Jul 17 13:54:26.902554 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:26.902504    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm" podStartSLOduration=0.661380612 podStartE2EDuration="7.902486558s" totalImagesPullingTime="7.241105946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:54:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:54:26.900930475 +0000 UTC m=+552.607377300" watchObservedRunningTime="2026-07-17 13:54:26.902486558 +0000 UTC m=+552.608933395"
Jul 17 13:54:27.391632 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:27.391600    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:27.396027 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:27.395978    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:27.884587 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:27.884549    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw" event={"ID":"a500635e-b087-4d94-888b-bada40827251","Type":"ContainerStarted","Data":"7a7e98d79196eb7e584b16a1fdef093bf84257cf2bc231a3c9c1ace9df4e7f0e"}
Jul 17 13:54:27.885085 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:27.884730    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:54:27.886093 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:27.886070    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv" event={"ID":"0a275428-cc40-4134-9ed6-0f48a8c9d67c","Type":"ContainerStarted","Data":"55c55b71a456c907bcc09d9c7a5eb33c98865798ec401e94770a0a462e073766"}
Jul 17 13:54:27.886320 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:27.886302    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:27.886392 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:27.886342    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"
Jul 17 13:54:27.887124 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:27.887105    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ftdhm"
Jul 17 13:54:27.902289 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:27.902242    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw" podStartSLOduration=2.710059779 podStartE2EDuration="6.902227742s" totalImagesPullingTime="4.192167963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:54:27.900851704 +0000 UTC m=+553.607298539" watchObservedRunningTime="2026-07-17 13:54:27.902227742 +0000 UTC m=+553.608674597"
Jul 17 13:54:27.936486 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:27.936428    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv" podStartSLOduration=2.170924203 podStartE2EDuration="6.936406247s" totalImagesPullingTime="4.765482044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:54:27.934450881 +0000 UTC m=+553.640897712" watchObservedRunningTime="2026-07-17 13:54:27.936406247 +0000 UTC m=+553.642853082"
Jul 17 13:54:58.893460 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:58.893428    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22dhv"
Jul 17 13:54:58.896788 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:54:58.896768    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hcggw"
Jul 17 13:55:10.082194 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.082159    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ljnh"]
Jul 17 13:55:10.082588 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.082545    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4745e924-9fb9-4358-9acf-3dc1db98b63e" containerName="discovery"
Jul 17 13:55:10.082588 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.082556    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4745e924-9fb9-4358-9acf-3dc1db98b63e" containerName="discovery"
Jul 17 13:55:10.082672 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.082614    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4745e924-9fb9-4358-9acf-3dc1db98b63e" containerName="discovery"
Jul 17 13:55:10.084637 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.084619    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ljnh"
Jul 17 13:55:10.087323 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.087291    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 13:55:10.087438 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.087386    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-zwglj\""
Jul 17 13:55:10.088160 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.088144    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 13:55:10.102125 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.102092    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ljnh"]
Jul 17 13:55:10.156173 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.156147    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snwng\" (UniqueName: \"kubernetes.io/projected/a688630a-7c3d-47ac-b8b5-f77dd4fe21a9-kube-api-access-snwng\") pod \"kuadrant-operator-catalog-5ljnh\" (UID: \"a688630a-7c3d-47ac-b8b5-f77dd4fe21a9\") " pod="kuadrant-system/kuadrant-operator-catalog-5ljnh"
Jul 17 13:55:10.257497 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.257462    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snwng\" (UniqueName: \"kubernetes.io/projected/a688630a-7c3d-47ac-b8b5-f77dd4fe21a9-kube-api-access-snwng\") pod \"kuadrant-operator-catalog-5ljnh\" (UID: \"a688630a-7c3d-47ac-b8b5-f77dd4fe21a9\") " pod="kuadrant-system/kuadrant-operator-catalog-5ljnh"
Jul 17 13:55:10.266402 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.266380    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snwng\" (UniqueName: \"kubernetes.io/projected/a688630a-7c3d-47ac-b8b5-f77dd4fe21a9-kube-api-access-snwng\") pod \"kuadrant-operator-catalog-5ljnh\" (UID: \"a688630a-7c3d-47ac-b8b5-f77dd4fe21a9\") " pod="kuadrant-system/kuadrant-operator-catalog-5ljnh"
Jul 17 13:55:10.393471 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.393404    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ljnh"
Jul 17 13:55:10.441540 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.441515    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ljnh"]
Jul 17 13:55:10.531121 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.531094    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ljnh"]
Jul 17 13:55:10.532744 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:55:10.532714    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda688630a_7c3d_47ac_b8b5_f77dd4fe21a9.slice/crio-41e927e28b517e8ea61bbc76f77764a17bcfaf3044e5ce66b5af5bea80eb9d25 WatchSource:0}: Error finding container 41e927e28b517e8ea61bbc76f77764a17bcfaf3044e5ce66b5af5bea80eb9d25: Status 404 returned error can't find the container with id 41e927e28b517e8ea61bbc76f77764a17bcfaf3044e5ce66b5af5bea80eb9d25
Jul 17 13:55:10.649762 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.649701    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t4mgn"]
Jul 17 13:55:10.652409 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.652391    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-t4mgn"
Jul 17 13:55:10.670636 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.670612    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t4mgn"]
Jul 17 13:55:10.761524 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.761493    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p4kv\" (UniqueName: \"kubernetes.io/projected/c9d86e80-afd0-40c9-a2cd-52795268301a-kube-api-access-8p4kv\") pod \"kuadrant-operator-catalog-t4mgn\" (UID: \"c9d86e80-afd0-40c9-a2cd-52795268301a\") " pod="kuadrant-system/kuadrant-operator-catalog-t4mgn"
Jul 17 13:55:10.862306 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.862272    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p4kv\" (UniqueName: \"kubernetes.io/projected/c9d86e80-afd0-40c9-a2cd-52795268301a-kube-api-access-8p4kv\") pod \"kuadrant-operator-catalog-t4mgn\" (UID: \"c9d86e80-afd0-40c9-a2cd-52795268301a\") " pod="kuadrant-system/kuadrant-operator-catalog-t4mgn"
Jul 17 13:55:10.878952 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.878926    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p4kv\" (UniqueName: \"kubernetes.io/projected/c9d86e80-afd0-40c9-a2cd-52795268301a-kube-api-access-8p4kv\") pod \"kuadrant-operator-catalog-t4mgn\" (UID: \"c9d86e80-afd0-40c9-a2cd-52795268301a\") " pod="kuadrant-system/kuadrant-operator-catalog-t4mgn"
Jul 17 13:55:10.961051 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:10.961027    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-t4mgn"
Jul 17 13:55:11.087721 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:11.087686    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ljnh" event={"ID":"a688630a-7c3d-47ac-b8b5-f77dd4fe21a9","Type":"ContainerStarted","Data":"41e927e28b517e8ea61bbc76f77764a17bcfaf3044e5ce66b5af5bea80eb9d25"}
Jul 17 13:55:11.128899 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:11.128874    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-t4mgn"]
Jul 17 13:55:11.132962 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:55:11.132937    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d86e80_afd0_40c9_a2cd_52795268301a.slice/crio-dfc110eb2ee81d8133a2a6cc99685aa528460a796bb591cc3162070ecbca4888 WatchSource:0}: Error finding container dfc110eb2ee81d8133a2a6cc99685aa528460a796bb591cc3162070ecbca4888: Status 404 returned error can't find the container with id dfc110eb2ee81d8133a2a6cc99685aa528460a796bb591cc3162070ecbca4888
Jul 17 13:55:12.096182 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:12.096106    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-t4mgn" event={"ID":"c9d86e80-afd0-40c9-a2cd-52795268301a","Type":"ContainerStarted","Data":"dfc110eb2ee81d8133a2a6cc99685aa528460a796bb591cc3162070ecbca4888"}
Jul 17 13:55:13.102509 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:13.102419    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-t4mgn" event={"ID":"c9d86e80-afd0-40c9-a2cd-52795268301a","Type":"ContainerStarted","Data":"089d5511acb275949812e20f856f29b98eb8acc8b7657ba9f591f9290d2b2708"}
Jul 17 13:55:13.103965 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:13.103940    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ljnh" event={"ID":"a688630a-7c3d-47ac-b8b5-f77dd4fe21a9","Type":"ContainerStarted","Data":"83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf"}
Jul 17 13:55:13.104097 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:13.104000    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5ljnh" podUID="a688630a-7c3d-47ac-b8b5-f77dd4fe21a9" containerName="registry-server" containerID="cri-o://83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf" gracePeriod=2
Jul 17 13:55:13.120246 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:13.120208    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-t4mgn" podStartSLOduration=1.444230154 podStartE2EDuration="3.120194431s" totalImagesPullingTime="1.675964277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:55:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:55:13.118261816 +0000 UTC m=+598.824708650" watchObservedRunningTime="2026-07-17 13:55:13.120194431 +0000 UTC m=+598.826641265"
Jul 17 13:55:13.134319 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:13.134282    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5ljnh" podStartSLOduration=0.861858497 podStartE2EDuration="3.134269495s" totalImagesPullingTime="2.272410998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:55:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:55:13.133316677 +0000 UTC m=+598.839763529" watchObservedRunningTime="2026-07-17 13:55:13.134269495 +0000 UTC m=+598.840716350"
Jul 17 13:55:13.358315 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:13.358265    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ljnh"
Jul 17 13:55:13.488174 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:13.488145    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snwng\" (UniqueName: \"kubernetes.io/projected/a688630a-7c3d-47ac-b8b5-f77dd4fe21a9-kube-api-access-snwng\") pod \"a688630a-7c3d-47ac-b8b5-f77dd4fe21a9\" (UID: \"a688630a-7c3d-47ac-b8b5-f77dd4fe21a9\") "
Jul 17 13:55:13.490385 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:13.490355    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a688630a-7c3d-47ac-b8b5-f77dd4fe21a9-kube-api-access-snwng" (OuterVolumeSpecName: "kube-api-access-snwng") pod "a688630a-7c3d-47ac-b8b5-f77dd4fe21a9" (UID: "a688630a-7c3d-47ac-b8b5-f77dd4fe21a9"). InnerVolumeSpecName "kube-api-access-snwng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:55:13.589653 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:13.589629    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snwng\" (UniqueName: \"kubernetes.io/projected/a688630a-7c3d-47ac-b8b5-f77dd4fe21a9-kube-api-access-snwng\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:55:14.110113 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:14.110039    2561 generic.go:356] "Generic (PLEG): container finished" podID="a688630a-7c3d-47ac-b8b5-f77dd4fe21a9" containerID="83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf" exitCode=0
Jul 17 13:55:14.110113 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:14.110078    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ljnh"
Jul 17 13:55:14.110113 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:14.110101    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ljnh" event={"ID":"a688630a-7c3d-47ac-b8b5-f77dd4fe21a9","Type":"ContainerDied","Data":"83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf"}
Jul 17 13:55:14.110539 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:14.110147    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ljnh" event={"ID":"a688630a-7c3d-47ac-b8b5-f77dd4fe21a9","Type":"ContainerDied","Data":"41e927e28b517e8ea61bbc76f77764a17bcfaf3044e5ce66b5af5bea80eb9d25"}
Jul 17 13:55:14.110539 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:14.110167    2561 scope.go:117] "RemoveContainer" containerID="83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf"
Jul 17 13:55:14.119328 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:14.119312    2561 scope.go:117] "RemoveContainer" containerID="83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf"
Jul 17 13:55:14.119596 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:55:14.119580    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf\": container with ID starting with 83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf not found: ID does not exist" containerID="83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf"
Jul 17 13:55:14.119651 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:14.119606    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf"} err="failed to get container status \"83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf\": rpc error: code = NotFound desc = could not find container \"83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf\": container with ID starting with 83e4fbe7ae5921ca24b4a21a655c4c24a977aa8b10863cede15a73a20144ccdf not found: ID does not exist"
Jul 17 13:55:14.131132 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:14.131113    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ljnh"]
Jul 17 13:55:14.133495 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:14.133475    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ljnh"]
Jul 17 13:55:14.772069 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:14.772039    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a688630a-7c3d-47ac-b8b5-f77dd4fe21a9" path="/var/lib/kubelet/pods/a688630a-7c3d-47ac-b8b5-f77dd4fe21a9/volumes"
Jul 17 13:55:20.961528 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:20.961500    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-t4mgn"
Jul 17 13:55:20.963969 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:20.961556    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-t4mgn"
Jul 17 13:55:20.984418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:20.984399    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-t4mgn"
Jul 17 13:55:21.164656 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:21.164629    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-t4mgn"
Jul 17 13:55:36.801285 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:36.801194    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bgzng"]
Jul 17 13:55:36.801851 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:36.801732    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a688630a-7c3d-47ac-b8b5-f77dd4fe21a9" containerName="registry-server"
Jul 17 13:55:36.801851 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:36.801750    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a688630a-7c3d-47ac-b8b5-f77dd4fe21a9" containerName="registry-server"
Jul 17 13:55:36.801965 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:36.801879    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a688630a-7c3d-47ac-b8b5-f77dd4fe21a9" containerName="registry-server"
Jul 17 13:55:36.804949 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:36.804929    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bgzng"
Jul 17 13:55:36.808179 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:36.808155    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-x2wkg\""
Jul 17 13:55:36.818964 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:36.818935    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bgzng"]
Jul 17 13:55:36.885059 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:36.884996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv2m6\" (UniqueName: \"kubernetes.io/projected/489c3b55-dbf8-4327-90cd-d97b261a24df-kube-api-access-nv2m6\") pod \"authorino-operator-75cc8d5cb7-bgzng\" (UID: \"489c3b55-dbf8-4327-90cd-d97b261a24df\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bgzng"
Jul 17 13:55:36.986679 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:36.986646    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv2m6\" (UniqueName: \"kubernetes.io/projected/489c3b55-dbf8-4327-90cd-d97b261a24df-kube-api-access-nv2m6\") pod \"authorino-operator-75cc8d5cb7-bgzng\" (UID: \"489c3b55-dbf8-4327-90cd-d97b261a24df\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bgzng"
Jul 17 13:55:37.007481 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:37.007456    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv2m6\" (UniqueName: \"kubernetes.io/projected/489c3b55-dbf8-4327-90cd-d97b261a24df-kube-api-access-nv2m6\") pod \"authorino-operator-75cc8d5cb7-bgzng\" (UID: \"489c3b55-dbf8-4327-90cd-d97b261a24df\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bgzng"
Jul 17 13:55:37.114945 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:37.114858    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bgzng"
Jul 17 13:55:37.285034 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:37.284989    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bgzng"]
Jul 17 13:55:37.288665 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:55:37.288617    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489c3b55_dbf8_4327_90cd_d97b261a24df.slice/crio-055d7472f9c25d6a61129035dcc00e6b99e466da8163780e18e52c4b086a4ed2 WatchSource:0}: Error finding container 055d7472f9c25d6a61129035dcc00e6b99e466da8163780e18e52c4b086a4ed2: Status 404 returned error can't find the container with id 055d7472f9c25d6a61129035dcc00e6b99e466da8163780e18e52c4b086a4ed2
Jul 17 13:55:37.291041 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:37.290994    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 13:55:38.230576 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:38.230536    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bgzng" event={"ID":"489c3b55-dbf8-4327-90cd-d97b261a24df","Type":"ContainerStarted","Data":"055d7472f9c25d6a61129035dcc00e6b99e466da8163780e18e52c4b086a4ed2"}
Jul 17 13:55:40.031830 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.031790    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj"]
Jul 17 13:55:40.035460 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.035444    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj"
Jul 17 13:55:40.047349 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.047327    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 17 13:55:40.051148 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.051131    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-8qjfs\""
Jul 17 13:55:40.057409 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.057381    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj"]
Jul 17 13:55:40.219291 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.219247    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cm2l\" (UniqueName: \"kubernetes.io/projected/f9d5b96a-e2a7-4af4-8595-4e517357057a-kube-api-access-9cm2l\") pod \"dns-operator-controller-manager-864dd78c4c-f5gnj\" (UID: \"f9d5b96a-e2a7-4af4-8595-4e517357057a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj"
Jul 17 13:55:40.243966 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.243922    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bgzng" event={"ID":"489c3b55-dbf8-4327-90cd-d97b261a24df","Type":"ContainerStarted","Data":"a599748139d45c230cecb7291a9c2e1c5b83e6a967234c30bbaf1e0aa1102b18"}
Jul 17 13:55:40.244187 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.244022    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bgzng"
Jul 17 13:55:40.266786 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.266730    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bgzng" podStartSLOduration=2.054705209 podStartE2EDuration="4.266711636s" totalImagesPullingTime="2.212006427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:55:40.265830223 +0000 UTC m=+625.972277056" watchObservedRunningTime="2026-07-17 13:55:40.266711636 +0000 UTC m=+625.973158470"
Jul 17 13:55:40.320500 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.320418    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cm2l\" (UniqueName: \"kubernetes.io/projected/f9d5b96a-e2a7-4af4-8595-4e517357057a-kube-api-access-9cm2l\") pod \"dns-operator-controller-manager-864dd78c4c-f5gnj\" (UID: \"f9d5b96a-e2a7-4af4-8595-4e517357057a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj"
Jul 17 13:55:40.333543 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.333516    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cm2l\" (UniqueName: \"kubernetes.io/projected/f9d5b96a-e2a7-4af4-8595-4e517357057a-kube-api-access-9cm2l\") pod \"dns-operator-controller-manager-864dd78c4c-f5gnj\" (UID: \"f9d5b96a-e2a7-4af4-8595-4e517357057a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj"
Jul 17 13:55:40.345325 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.345300    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj"
Jul 17 13:55:40.509893 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:40.509866    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj"]
Jul 17 13:55:40.511675 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:55:40.511651    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d5b96a_e2a7_4af4_8595_4e517357057a.slice/crio-abf7f287d9ac2f4a047d9605e26f61310e62929295cb8772229170dc95c2fcf1 WatchSource:0}: Error finding container abf7f287d9ac2f4a047d9605e26f61310e62929295cb8772229170dc95c2fcf1: Status 404 returned error can't find the container with id abf7f287d9ac2f4a047d9605e26f61310e62929295cb8772229170dc95c2fcf1
Jul 17 13:55:41.250900 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:41.250855    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj" event={"ID":"f9d5b96a-e2a7-4af4-8595-4e517357057a","Type":"ContainerStarted","Data":"abf7f287d9ac2f4a047d9605e26f61310e62929295cb8772229170dc95c2fcf1"}
Jul 17 13:55:42.257406 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:42.257374    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj" event={"ID":"f9d5b96a-e2a7-4af4-8595-4e517357057a","Type":"ContainerStarted","Data":"2601ad0141452ddad3cfd2e68cbe0c8f1d2bb47b391ed613fd50804b5238186a"}
Jul 17 13:55:42.257747 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:42.257486    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj"
Jul 17 13:55:42.289802 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:42.289758    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj" podStartSLOduration=0.650840091 podStartE2EDuration="2.289743977s" totalImagesPullingTime="1.638903886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:55:42.288106097 +0000 UTC m=+627.994552930" watchObservedRunningTime="2026-07-17 13:55:42.289743977 +0000 UTC m=+627.996190810"
Jul 17 13:55:42.728125 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:42.728094    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"]
Jul 17 13:55:42.731647 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:42.731625    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:55:42.737061 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:42.737041    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hshhm\""
Jul 17 13:55:42.752313 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:42.752292    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"]
Jul 17 13:55:42.845406 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:42.845378    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27cmj\" (UniqueName: \"kubernetes.io/projected/9f2c0a42-908f-40c8-a9c4-4186ebe68389-kube-api-access-27cmj\") pod \"limitador-operator-controller-manager-859d7b78bd-vht22\" (UID: \"9f2c0a42-908f-40c8-a9c4-4186ebe68389\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:55:42.946341 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:42.946310    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27cmj\" (UniqueName: \"kubernetes.io/projected/9f2c0a42-908f-40c8-a9c4-4186ebe68389-kube-api-access-27cmj\") pod \"limitador-operator-controller-manager-859d7b78bd-vht22\" (UID: \"9f2c0a42-908f-40c8-a9c4-4186ebe68389\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:55:42.973339 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:42.973310    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27cmj\" (UniqueName: \"kubernetes.io/projected/9f2c0a42-908f-40c8-a9c4-4186ebe68389-kube-api-access-27cmj\") pod \"limitador-operator-controller-manager-859d7b78bd-vht22\" (UID: \"9f2c0a42-908f-40c8-a9c4-4186ebe68389\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:55:43.041396 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:43.041339    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:55:43.231539 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:55:43.231509    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f2c0a42_908f_40c8_a9c4_4186ebe68389.slice/crio-66e6bb5e8eff5cb94c5c4022fe8c30a08df0d5c7a3ab8c206237af1d0af5f663 WatchSource:0}: Error finding container 66e6bb5e8eff5cb94c5c4022fe8c30a08df0d5c7a3ab8c206237af1d0af5f663: Status 404 returned error can't find the container with id 66e6bb5e8eff5cb94c5c4022fe8c30a08df0d5c7a3ab8c206237af1d0af5f663
Jul 17 13:55:43.249941 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:43.249867    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"]
Jul 17 13:55:43.267054 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:43.267024    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22" event={"ID":"9f2c0a42-908f-40c8-a9c4-4186ebe68389","Type":"ContainerStarted","Data":"66e6bb5e8eff5cb94c5c4022fe8c30a08df0d5c7a3ab8c206237af1d0af5f663"}
Jul 17 13:55:45.278398 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:45.278366    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22" event={"ID":"9f2c0a42-908f-40c8-a9c4-4186ebe68389","Type":"ContainerStarted","Data":"adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d"}
Jul 17 13:55:45.278776 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:45.278518    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:55:45.300731 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:45.300689    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22" podStartSLOduration=1.7994633530000002 podStartE2EDuration="3.300675217s" totalImagesPullingTime="1.501211864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:55:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:55:45.298508292 +0000 UTC m=+631.004955125" watchObservedRunningTime="2026-07-17 13:55:45.300675217 +0000 UTC m=+631.007122051"
Jul 17 13:55:47.093071 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.093040    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"]
Jul 17 13:55:47.101562 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.101503    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:55:47.104880 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.104863    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bzpxn\""
Jul 17 13:55:47.119372 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.119348    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"]
Jul 17 13:55:47.285773 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.285744    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" (UID: \"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:55:47.285928 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.285782    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x6br\" (UniqueName: \"kubernetes.io/projected/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-kube-api-access-7x6br\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" (UID: \"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:55:47.386422 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.386354    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" (UID: \"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:55:47.386422 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.386392    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x6br\" (UniqueName: \"kubernetes.io/projected/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-kube-api-access-7x6br\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" (UID: \"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:55:47.386737 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.386717    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" (UID: \"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:55:47.398815 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.398787    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x6br\" (UniqueName: \"kubernetes.io/projected/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-kube-api-access-7x6br\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" (UID: \"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:55:47.411562 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.411536    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:55:47.554910 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.554882    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"]
Jul 17 13:55:47.557552 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:55:47.557511    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c075a15_3bb6_4eed_9bb5_a3fd886a0c17.slice/crio-b13dbd7692106d1f8ca5da301c7bb7600a93afeb35c186f45b97d8bb5df6e9c6 WatchSource:0}: Error finding container b13dbd7692106d1f8ca5da301c7bb7600a93afeb35c186f45b97d8bb5df6e9c6: Status 404 returned error can't find the container with id b13dbd7692106d1f8ca5da301c7bb7600a93afeb35c186f45b97d8bb5df6e9c6
Jul 17 13:55:47.725406 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.725380    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-894c5b5cf-6m2kf"]
Jul 17 13:55:47.730967 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.730948    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.740130 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.740105    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-894c5b5cf-6m2kf"]
Jul 17 13:55:47.891330 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.891302    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-service-ca\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.891452 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.891343    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-console-oauth-config\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.891452 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.891371    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfvjd\" (UniqueName: \"kubernetes.io/projected/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-kube-api-access-zfvjd\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.891525 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.891496    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-console-serving-cert\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.891525 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.891518    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-oauth-serving-cert\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.891601 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.891588    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-trusted-ca-bundle\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.891636 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.891627    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-console-config\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.992327 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.992263    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfvjd\" (UniqueName: \"kubernetes.io/projected/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-kube-api-access-zfvjd\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.992327 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.992317    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-console-serving-cert\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.992469 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.992426    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-oauth-serving-cert\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.992586 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.992490    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-trusted-ca-bundle\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.992586 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.992535    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-console-config\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.992586 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.992560    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-service-ca\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.992743 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.992588    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-console-oauth-config\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.993295 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.993236    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-oauth-serving-cert\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.993411 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.993333    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-console-config\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.993489 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.993471    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-trusted-ca-bundle\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.993580 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.993557    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-service-ca\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.994997 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.994974    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-console-serving-cert\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:47.995099 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:47.995035    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-console-oauth-config\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:48.001719 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:48.001703    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfvjd\" (UniqueName: \"kubernetes.io/projected/cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1-kube-api-access-zfvjd\") pod \"console-894c5b5cf-6m2kf\" (UID: \"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1\") " pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:48.041642 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:48.041622    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:48.213167 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:48.213135    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-894c5b5cf-6m2kf"]
Jul 17 13:55:48.217552 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:55:48.217524    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf24f6a4_cc73_4636_8b5b_e57a18a4f7c1.slice/crio-0991bc6889b392175b2d8e0942b4163116d6d89b4c193c19aca83f690f7b03eb WatchSource:0}: Error finding container 0991bc6889b392175b2d8e0942b4163116d6d89b4c193c19aca83f690f7b03eb: Status 404 returned error can't find the container with id 0991bc6889b392175b2d8e0942b4163116d6d89b4c193c19aca83f690f7b03eb
Jul 17 13:55:48.296410 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:48.296377    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-894c5b5cf-6m2kf" event={"ID":"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1","Type":"ContainerStarted","Data":"0991bc6889b392175b2d8e0942b4163116d6d89b4c193c19aca83f690f7b03eb"}
Jul 17 13:55:48.297782 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:48.297757    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp" event={"ID":"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17","Type":"ContainerStarted","Data":"b13dbd7692106d1f8ca5da301c7bb7600a93afeb35c186f45b97d8bb5df6e9c6"}
Jul 17 13:55:49.304541 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:49.304504    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-894c5b5cf-6m2kf" event={"ID":"cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1","Type":"ContainerStarted","Data":"eeef0c97159cbfb9a43eae3437000f1c8d8eb9beec1b7277243e587c3894f8b2"}
Jul 17 13:55:49.327621 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:49.327567    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-894c5b5cf-6m2kf" podStartSLOduration=2.327546794 podStartE2EDuration="2.327546794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:55:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:55:49.325254563 +0000 UTC m=+635.031701397" watchObservedRunningTime="2026-07-17 13:55:49.327546794 +0000 UTC m=+635.033993630"
Jul 17 13:55:51.254082 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:51.254042    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bgzng"
Jul 17 13:55:53.269691 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:53.269664    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-f5gnj"
Jul 17 13:55:53.332544 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:53.332511    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp" event={"ID":"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17","Type":"ContainerStarted","Data":"6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e"}
Jul 17 13:55:53.332720 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:53.332658    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:55:53.352026 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:53.351952    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp" podStartSLOduration=1.5213240730000002 podStartE2EDuration="6.35193727s" totalImagesPullingTime="4.830613197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:55:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:55:53.350907037 +0000 UTC m=+639.057353893" watchObservedRunningTime="2026-07-17 13:55:53.35193727 +0000 UTC m=+639.058384104"
Jul 17 13:55:56.288092 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:56.288065    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:55:58.042122 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:58.042090    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:58.042585 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:58.042160    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:58.046608 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:58.046586    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:58.362725 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:58.362645    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-894c5b5cf-6m2kf"
Jul 17 13:55:58.415154 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:55:58.415118    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c44494d47-7g9f6"]
Jul 17 13:56:04.340952 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:04.340916    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:56:06.305263 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.305232    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"]
Jul 17 13:56:06.305628 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.305422    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp" podUID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" containerName="manager" containerID="cri-o://6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e" gracePeriod=2
Jul 17 13:56:06.315426 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.315401    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"]
Jul 17 13:56:06.322621 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:06.322574    2561 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\": pods \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:56:06.331792 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.331766    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"]
Jul 17 13:56:06.332078 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.332028    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22" podUID="9f2c0a42-908f-40c8-a9c4-4186ebe68389" containerName="manager" containerID="cri-o://adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d" gracePeriod=2
Jul 17 13:56:06.339128 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.339103    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"]
Jul 17 13:56:06.339726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.339704    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" containerName="manager"
Jul 17 13:56:06.339870 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.339753    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" containerName="manager"
Jul 17 13:56:06.339961 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.339937    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" containerName="manager"
Jul 17 13:56:06.343346 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.343323    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"]
Jul 17 13:56:06.343478 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.343427    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:06.345575 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:06.345546    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:56:06.365775 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.365745    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"]
Jul 17 13:56:06.387598 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:06.387533    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vht22\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="9f2c0a42-908f-40c8-a9c4-4186ebe68389" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:56:06.387763 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.387545    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd"]
Jul 17 13:56:06.388154 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.388136    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f2c0a42-908f-40c8-a9c4-4186ebe68389" containerName="manager"
Jul 17 13:56:06.388154 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.388154    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f2c0a42-908f-40c8-a9c4-4186ebe68389" containerName="manager"
Jul 17 13:56:06.388296 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.388278    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f2c0a42-908f-40c8-a9c4-4186ebe68389" containerName="manager"
Jul 17 13:56:06.390162 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:06.390120    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:56:06.391803 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.391789    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd"
Jul 17 13:56:06.407685 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.407655    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd"]
Jul 17 13:56:06.428928 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:06.428899    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vht22\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="9f2c0a42-908f-40c8-a9c4-4186ebe68389" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:56:06.431215 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:06.431187    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:56:06.445225 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.445200    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/197f9af6-da6f-4eb9-9dfd-83101e8adc13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-66qfc\" (UID: \"197f9af6-da6f-4eb9-9dfd-83101e8adc13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:06.445499 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.445250    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7t7r\" (UniqueName: \"kubernetes.io/projected/197f9af6-da6f-4eb9-9dfd-83101e8adc13-kube-api-access-w7t7r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-66qfc\" (UID: \"197f9af6-da6f-4eb9-9dfd-83101e8adc13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:06.445499 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.445268    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68kvm\" (UniqueName: \"kubernetes.io/projected/8b0faee6-d205-4fcc-9172-5f70ffb4df39-kube-api-access-68kvm\") pod \"limitador-operator-controller-manager-859d7b78bd-l6frd\" (UID: \"8b0faee6-d205-4fcc-9172-5f70ffb4df39\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd"
Jul 17 13:56:06.546758 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.546728    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/197f9af6-da6f-4eb9-9dfd-83101e8adc13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-66qfc\" (UID: \"197f9af6-da6f-4eb9-9dfd-83101e8adc13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:06.546936 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.546777    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7t7r\" (UniqueName: \"kubernetes.io/projected/197f9af6-da6f-4eb9-9dfd-83101e8adc13-kube-api-access-w7t7r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-66qfc\" (UID: \"197f9af6-da6f-4eb9-9dfd-83101e8adc13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:06.546936 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.546800    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68kvm\" (UniqueName: \"kubernetes.io/projected/8b0faee6-d205-4fcc-9172-5f70ffb4df39-kube-api-access-68kvm\") pod \"limitador-operator-controller-manager-859d7b78bd-l6frd\" (UID: \"8b0faee6-d205-4fcc-9172-5f70ffb4df39\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd"
Jul 17 13:56:06.547216 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.547192    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/197f9af6-da6f-4eb9-9dfd-83101e8adc13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-66qfc\" (UID: \"197f9af6-da6f-4eb9-9dfd-83101e8adc13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:06.582643 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.582556    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68kvm\" (UniqueName: \"kubernetes.io/projected/8b0faee6-d205-4fcc-9172-5f70ffb4df39-kube-api-access-68kvm\") pod \"limitador-operator-controller-manager-859d7b78bd-l6frd\" (UID: \"8b0faee6-d205-4fcc-9172-5f70ffb4df39\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd"
Jul 17 13:56:06.584592 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.584567    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7t7r\" (UniqueName: \"kubernetes.io/projected/197f9af6-da6f-4eb9-9dfd-83101e8adc13-kube-api-access-w7t7r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-66qfc\" (UID: \"197f9af6-da6f-4eb9-9dfd-83101e8adc13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:06.611072 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.611049    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:56:06.613068 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.613051    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:56:06.613304 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:06.613281    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vht22\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="9f2c0a42-908f-40c8-a9c4-4186ebe68389" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:56:06.615356 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:06.615339    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:56:06.617354 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:06.617333    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vht22\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="9f2c0a42-908f-40c8-a9c4-4186ebe68389" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:56:06.619149 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:06.619127    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4nqtp\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:56:06.647587 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.647566    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27cmj\" (UniqueName: \"kubernetes.io/projected/9f2c0a42-908f-40c8-a9c4-4186ebe68389-kube-api-access-27cmj\") pod \"9f2c0a42-908f-40c8-a9c4-4186ebe68389\" (UID: \"9f2c0a42-908f-40c8-a9c4-4186ebe68389\") "
Jul 17 13:56:06.647666 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.647618    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-extensions-socket-volume\") pod \"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\" (UID: \"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\") "
Jul 17 13:56:06.647666 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.647649    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x6br\" (UniqueName: \"kubernetes.io/projected/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-kube-api-access-7x6br\") pod \"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\" (UID: \"7c075a15-3bb6-4eed-9bb5-a3fd886a0c17\") "
Jul 17 13:56:06.648189 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.648129    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" (UID: "7c075a15-3bb6-4eed-9bb5-a3fd886a0c17"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 13:56:06.649610 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.649592    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f2c0a42-908f-40c8-a9c4-4186ebe68389-kube-api-access-27cmj" (OuterVolumeSpecName: "kube-api-access-27cmj") pod "9f2c0a42-908f-40c8-a9c4-4186ebe68389" (UID: "9f2c0a42-908f-40c8-a9c4-4186ebe68389"). InnerVolumeSpecName "kube-api-access-27cmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:56:06.649685 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.649661    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-kube-api-access-7x6br" (OuterVolumeSpecName: "kube-api-access-7x6br") pod "7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" (UID: "7c075a15-3bb6-4eed-9bb5-a3fd886a0c17"). InnerVolumeSpecName "kube-api-access-7x6br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:56:06.746793 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.746764    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:06.749223 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.749201    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27cmj\" (UniqueName: \"kubernetes.io/projected/9f2c0a42-908f-40c8-a9c4-4186ebe68389-kube-api-access-27cmj\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:06.749303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.749225    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-extensions-socket-volume\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:06.749303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.749235    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x6br\" (UniqueName: \"kubernetes.io/projected/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17-kube-api-access-7x6br\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:06.752712 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.752689    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd"
Jul 17 13:56:06.773852 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.773817    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" path="/var/lib/kubelet/pods/7c075a15-3bb6-4eed-9bb5-a3fd886a0c17/volumes"
Jul 17 13:56:06.774307 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.774285    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f2c0a42-908f-40c8-a9c4-4186ebe68389" path="/var/lib/kubelet/pods/9f2c0a42-908f-40c8-a9c4-4186ebe68389/volumes"
Jul 17 13:56:06.911601 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.911579    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd"]
Jul 17 13:56:06.915290 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:56:06.915267    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b0faee6_d205_4fcc_9172_5f70ffb4df39.slice/crio-31ea72b87d0b17b336cbbd100d3bde0b17fcceb393c94699e72e6a8a9e90bece WatchSource:0}: Error finding container 31ea72b87d0b17b336cbbd100d3bde0b17fcceb393c94699e72e6a8a9e90bece: Status 404 returned error can't find the container with id 31ea72b87d0b17b336cbbd100d3bde0b17fcceb393c94699e72e6a8a9e90bece
Jul 17 13:56:06.933266 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:06.933245    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"]
Jul 17 13:56:06.934525 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:56:06.934504    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod197f9af6_da6f_4eb9_9dfd_83101e8adc13.slice/crio-2ea1e4e2cbc9a5c86bfebad72007b76525f404c47f75b2ea8ea4ecea6ffac8e0 WatchSource:0}: Error finding container 2ea1e4e2cbc9a5c86bfebad72007b76525f404c47f75b2ea8ea4ecea6ffac8e0: Status 404 returned error can't find the container with id 2ea1e4e2cbc9a5c86bfebad72007b76525f404c47f75b2ea8ea4ecea6ffac8e0
Jul 17 13:56:07.406732 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.406696    2561 generic.go:356] "Generic (PLEG): container finished" podID="9f2c0a42-908f-40c8-a9c4-4186ebe68389" containerID="adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d" exitCode=0
Jul 17 13:56:07.407187 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.406745    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vht22"
Jul 17 13:56:07.407187 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.406810    2561 scope.go:117] "RemoveContainer" containerID="adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d"
Jul 17 13:56:07.408368 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.408342    2561 generic.go:356] "Generic (PLEG): container finished" podID="7c075a15-3bb6-4eed-9bb5-a3fd886a0c17" containerID="6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e" exitCode=0
Jul 17 13:56:07.408467 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.408378    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4nqtp"
Jul 17 13:56:07.410364 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.410338    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd" event={"ID":"8b0faee6-d205-4fcc-9172-5f70ffb4df39","Type":"ContainerStarted","Data":"04616219fb20cd2382c97b08501f8d8aee803b73c8bd4f66b9771e371e6b95e2"}
Jul 17 13:56:07.410531 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.410514    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd" event={"ID":"8b0faee6-d205-4fcc-9172-5f70ffb4df39","Type":"ContainerStarted","Data":"31ea72b87d0b17b336cbbd100d3bde0b17fcceb393c94699e72e6a8a9e90bece"}
Jul 17 13:56:07.410677 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.410662    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd"
Jul 17 13:56:07.412367 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.412346    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc" event={"ID":"197f9af6-da6f-4eb9-9dfd-83101e8adc13","Type":"ContainerStarted","Data":"5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f"}
Jul 17 13:56:07.412468 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.412390    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc" event={"ID":"197f9af6-da6f-4eb9-9dfd-83101e8adc13","Type":"ContainerStarted","Data":"2ea1e4e2cbc9a5c86bfebad72007b76525f404c47f75b2ea8ea4ecea6ffac8e0"}
Jul 17 13:56:07.412468 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.412418    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:07.416364 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.416349    2561 scope.go:117] "RemoveContainer" containerID="adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d"
Jul 17 13:56:07.416624 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:07.416606    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d\": container with ID starting with adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d not found: ID does not exist" containerID="adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d"
Jul 17 13:56:07.416697 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.416633    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d"} err="failed to get container status \"adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d\": rpc error: code = NotFound desc = could not find container \"adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d\": container with ID starting with adb578e3654452a5c5594354f6856404c0a1a6e9987a6112a6102f3d51825e3d not found: ID does not exist"
Jul 17 13:56:07.416697 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.416650    2561 scope.go:117] "RemoveContainer" containerID="6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e"
Jul 17 13:56:07.425377 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.425356    2561 scope.go:117] "RemoveContainer" containerID="6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e"
Jul 17 13:56:07.425666 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:07.425646    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e\": container with ID starting with 6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e not found: ID does not exist" containerID="6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e"
Jul 17 13:56:07.425721 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.425670    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e"} err="failed to get container status \"6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e\": rpc error: code = NotFound desc = could not find container \"6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e\": container with ID starting with 6b3de4b9c5a05fb3741201bc7211565bdc39e5abb079713db1aabd827d45265e not found: ID does not exist"
Jul 17 13:56:07.432473 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.432425    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd" podStartSLOduration=1.4324095159999999 podStartE2EDuration="1.432409516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:56:07.432214735 +0000 UTC m=+653.138661598" watchObservedRunningTime="2026-07-17 13:56:07.432409516 +0000 UTC m=+653.138856350"
Jul 17 13:56:07.451034 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:07.450983    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc" podStartSLOduration=1.45097399 podStartE2EDuration="1.45097399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:56:07.448603244 +0000 UTC m=+653.155050078" watchObservedRunningTime="2026-07-17 13:56:07.45097399 +0000 UTC m=+653.157420823"
Jul 17 13:56:18.421914 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:18.421886    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:19.427518 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:19.427490    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l6frd"
Jul 17 13:56:21.606975 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:21.606944    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"]
Jul 17 13:56:21.607406 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:21.607144    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc" podUID="197f9af6-da6f-4eb9-9dfd-83101e8adc13" containerName="manager" containerID="cri-o://5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f" gracePeriod=10
Jul 17 13:56:21.863813 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:21.863761    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:21.970655 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:21.970624    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7t7r\" (UniqueName: \"kubernetes.io/projected/197f9af6-da6f-4eb9-9dfd-83101e8adc13-kube-api-access-w7t7r\") pod \"197f9af6-da6f-4eb9-9dfd-83101e8adc13\" (UID: \"197f9af6-da6f-4eb9-9dfd-83101e8adc13\") "
Jul 17 13:56:21.970791 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:21.970675    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/197f9af6-da6f-4eb9-9dfd-83101e8adc13-extensions-socket-volume\") pod \"197f9af6-da6f-4eb9-9dfd-83101e8adc13\" (UID: \"197f9af6-da6f-4eb9-9dfd-83101e8adc13\") "
Jul 17 13:56:21.971088 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:21.971068    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/197f9af6-da6f-4eb9-9dfd-83101e8adc13-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "197f9af6-da6f-4eb9-9dfd-83101e8adc13" (UID: "197f9af6-da6f-4eb9-9dfd-83101e8adc13"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 13:56:21.972760 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:21.972745    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/197f9af6-da6f-4eb9-9dfd-83101e8adc13-kube-api-access-w7t7r" (OuterVolumeSpecName: "kube-api-access-w7t7r") pod "197f9af6-da6f-4eb9-9dfd-83101e8adc13" (UID: "197f9af6-da6f-4eb9-9dfd-83101e8adc13"). InnerVolumeSpecName "kube-api-access-w7t7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:56:22.039438 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.039405    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"]
Jul 17 13:56:22.039802 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.039790    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="197f9af6-da6f-4eb9-9dfd-83101e8adc13" containerName="manager"
Jul 17 13:56:22.039854 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.039803    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="197f9af6-da6f-4eb9-9dfd-83101e8adc13" containerName="manager"
Jul 17 13:56:22.039899 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.039890    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="197f9af6-da6f-4eb9-9dfd-83101e8adc13" containerName="manager"
Jul 17 13:56:22.043249 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.043231    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 13:56:22.071455 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.071428    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7t7r\" (UniqueName: \"kubernetes.io/projected/197f9af6-da6f-4eb9-9dfd-83101e8adc13-kube-api-access-w7t7r\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:22.071455 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.071455    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/197f9af6-da6f-4eb9-9dfd-83101e8adc13-extensions-socket-volume\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:22.076067 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.076045    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"]
Jul 17 13:56:22.172267 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.172235    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7kcq\" (UniqueName: \"kubernetes.io/projected/60875591-4314-475a-9258-4bd9bba42999-kube-api-access-k7kcq\") pod \"kuadrant-operator-controller-manager-86878c769f-xtvmf\" (UID: \"60875591-4314-475a-9258-4bd9bba42999\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 13:56:22.172385 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.172284    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60875591-4314-475a-9258-4bd9bba42999-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xtvmf\" (UID: \"60875591-4314-475a-9258-4bd9bba42999\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 13:56:22.273540 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.273513    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7kcq\" (UniqueName: \"kubernetes.io/projected/60875591-4314-475a-9258-4bd9bba42999-kube-api-access-k7kcq\") pod \"kuadrant-operator-controller-manager-86878c769f-xtvmf\" (UID: \"60875591-4314-475a-9258-4bd9bba42999\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 13:56:22.273658 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.273554    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60875591-4314-475a-9258-4bd9bba42999-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xtvmf\" (UID: \"60875591-4314-475a-9258-4bd9bba42999\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 13:56:22.273869 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.273853    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60875591-4314-475a-9258-4bd9bba42999-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xtvmf\" (UID: \"60875591-4314-475a-9258-4bd9bba42999\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 13:56:22.295279 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.295252    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7kcq\" (UniqueName: \"kubernetes.io/projected/60875591-4314-475a-9258-4bd9bba42999-kube-api-access-k7kcq\") pod \"kuadrant-operator-controller-manager-86878c769f-xtvmf\" (UID: \"60875591-4314-475a-9258-4bd9bba42999\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 13:56:22.353417 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.353395    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 13:56:22.497768 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.497732    2561 generic.go:356] "Generic (PLEG): container finished" podID="197f9af6-da6f-4eb9-9dfd-83101e8adc13" containerID="5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f" exitCode=0
Jul 17 13:56:22.497949 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.497782    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"
Jul 17 13:56:22.497949 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.497807    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc" event={"ID":"197f9af6-da6f-4eb9-9dfd-83101e8adc13","Type":"ContainerDied","Data":"5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f"}
Jul 17 13:56:22.497949 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.497863    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc" event={"ID":"197f9af6-da6f-4eb9-9dfd-83101e8adc13","Type":"ContainerDied","Data":"2ea1e4e2cbc9a5c86bfebad72007b76525f404c47f75b2ea8ea4ecea6ffac8e0"}
Jul 17 13:56:22.497949 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.497892    2561 scope.go:117] "RemoveContainer" containerID="5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f"
Jul 17 13:56:22.506881 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.506862    2561 scope.go:117] "RemoveContainer" containerID="5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f"
Jul 17 13:56:22.507154 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:22.507136    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f\": container with ID starting with 5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f not found: ID does not exist" containerID="5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f"
Jul 17 13:56:22.507217 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.507162    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f"} err="failed to get container status \"5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f\": rpc error: code = NotFound desc = could not find container \"5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f\": container with ID starting with 5df426a2395b6f0ee92168be456c95d267004e1e583375e5854c131cc5b8254f not found: ID does not exist"
Jul 17 13:56:22.526196 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.526177    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"]
Jul 17 13:56:22.528659 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:56:22.528636    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60875591_4314_475a_9258_4bd9bba42999.slice/crio-d0e3f83844fdbe322bd7e9185e8fd15084dfa69a1f2a4191528bd978016eaeca WatchSource:0}: Error finding container d0e3f83844fdbe322bd7e9185e8fd15084dfa69a1f2a4191528bd978016eaeca: Status 404 returned error can't find the container with id d0e3f83844fdbe322bd7e9185e8fd15084dfa69a1f2a4191528bd978016eaeca
Jul 17 13:56:22.549257 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.549237    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"]
Jul 17 13:56:22.565474 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.565456    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-66qfc"]
Jul 17 13:56:22.772713 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:22.772639    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="197f9af6-da6f-4eb9-9dfd-83101e8adc13" path="/var/lib/kubelet/pods/197f9af6-da6f-4eb9-9dfd-83101e8adc13/volumes"
Jul 17 13:56:23.438903 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.438862    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c44494d47-7g9f6" podUID="80b27150-4ddd-467c-acec-c4fb66953e48" containerName="console" containerID="cri-o://652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d" gracePeriod=15
Jul 17 13:56:23.505571 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.505541    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf" event={"ID":"60875591-4314-475a-9258-4bd9bba42999","Type":"ContainerStarted","Data":"8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887"}
Jul 17 13:56:23.505756 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.505601    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf" event={"ID":"60875591-4314-475a-9258-4bd9bba42999","Type":"ContainerStarted","Data":"d0e3f83844fdbe322bd7e9185e8fd15084dfa69a1f2a4191528bd978016eaeca"}
Jul 17 13:56:23.505756 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.505636    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 13:56:23.619173 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.619120    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf" podStartSLOduration=1.619103945 podStartE2EDuration="1.619103945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:56:23.615649869 +0000 UTC m=+669.322096728" watchObservedRunningTime="2026-07-17 13:56:23.619103945 +0000 UTC m=+669.325550778"
Jul 17 13:56:23.700050 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.699984    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:56:23.786743 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.786714    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-oauth-config\") pod \"80b27150-4ddd-467c-acec-c4fb66953e48\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") "
Jul 17 13:56:23.787107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.786756    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-console-config\") pod \"80b27150-4ddd-467c-acec-c4fb66953e48\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") "
Jul 17 13:56:23.787107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.786805    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-trusted-ca-bundle\") pod \"80b27150-4ddd-467c-acec-c4fb66953e48\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") "
Jul 17 13:56:23.787107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.786871    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-oauth-serving-cert\") pod \"80b27150-4ddd-467c-acec-c4fb66953e48\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") "
Jul 17 13:56:23.787107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.786896    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-service-ca\") pod \"80b27150-4ddd-467c-acec-c4fb66953e48\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") "
Jul 17 13:56:23.787107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.786915    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfspn\" (UniqueName: \"kubernetes.io/projected/80b27150-4ddd-467c-acec-c4fb66953e48-kube-api-access-rfspn\") pod \"80b27150-4ddd-467c-acec-c4fb66953e48\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") "
Jul 17 13:56:23.787107 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.786948    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-serving-cert\") pod \"80b27150-4ddd-467c-acec-c4fb66953e48\" (UID: \"80b27150-4ddd-467c-acec-c4fb66953e48\") "
Jul 17 13:56:23.787424 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.787272    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-console-config" (OuterVolumeSpecName: "console-config") pod "80b27150-4ddd-467c-acec-c4fb66953e48" (UID: "80b27150-4ddd-467c-acec-c4fb66953e48"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:56:23.787424 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.787394    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-service-ca" (OuterVolumeSpecName: "service-ca") pod "80b27150-4ddd-467c-acec-c4fb66953e48" (UID: "80b27150-4ddd-467c-acec-c4fb66953e48"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:56:23.787542 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.787519    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "80b27150-4ddd-467c-acec-c4fb66953e48" (UID: "80b27150-4ddd-467c-acec-c4fb66953e48"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:56:23.787726 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.787673    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "80b27150-4ddd-467c-acec-c4fb66953e48" (UID: "80b27150-4ddd-467c-acec-c4fb66953e48"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 13:56:23.789021 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.788981    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "80b27150-4ddd-467c-acec-c4fb66953e48" (UID: "80b27150-4ddd-467c-acec-c4fb66953e48"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:56:23.789303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.789274    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80b27150-4ddd-467c-acec-c4fb66953e48-kube-api-access-rfspn" (OuterVolumeSpecName: "kube-api-access-rfspn") pod "80b27150-4ddd-467c-acec-c4fb66953e48" (UID: "80b27150-4ddd-467c-acec-c4fb66953e48"). InnerVolumeSpecName "kube-api-access-rfspn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:56:23.789303 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.789272    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "80b27150-4ddd-467c-acec-c4fb66953e48" (UID: "80b27150-4ddd-467c-acec-c4fb66953e48"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 13:56:23.887630 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.887603    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-oauth-config\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:23.887630 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.887625    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-console-config\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:23.887765 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.887635    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-trusted-ca-bundle\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:23.887765 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.887644    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-oauth-serving-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:23.887765 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.887653    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80b27150-4ddd-467c-acec-c4fb66953e48-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:23.887765 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.887660    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfspn\" (UniqueName: \"kubernetes.io/projected/80b27150-4ddd-467c-acec-c4fb66953e48-kube-api-access-rfspn\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:23.887765 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:23.887668    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b27150-4ddd-467c-acec-c4fb66953e48-console-serving-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:24.514020 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:24.513967    2561 generic.go:356] "Generic (PLEG): container finished" podID="80b27150-4ddd-467c-acec-c4fb66953e48" containerID="652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d" exitCode=0
Jul 17 13:56:24.514233 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:24.514036    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c44494d47-7g9f6"
Jul 17 13:56:24.514233 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:24.514051    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c44494d47-7g9f6" event={"ID":"80b27150-4ddd-467c-acec-c4fb66953e48","Type":"ContainerDied","Data":"652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d"}
Jul 17 13:56:24.514233 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:24.514120    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c44494d47-7g9f6" event={"ID":"80b27150-4ddd-467c-acec-c4fb66953e48","Type":"ContainerDied","Data":"c4003de28fae4cf7e0c2ad62ddbc9a83c290fcde120d128474e48ba75acbf768"}
Jul 17 13:56:24.514233 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:24.514142    2561 scope.go:117] "RemoveContainer" containerID="652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d"
Jul 17 13:56:24.523621 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:24.523602    2561 scope.go:117] "RemoveContainer" containerID="652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d"
Jul 17 13:56:24.523865 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:24.523844    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d\": container with ID starting with 652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d not found: ID does not exist" containerID="652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d"
Jul 17 13:56:24.523952 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:24.523875    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d"} err="failed to get container status \"652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d\": rpc error: code = NotFound desc = could not find container \"652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d\": container with ID starting with 652239f7a9a0b137ec84967768233432b84bc822c0be97e50b7000f8995cf75d not found: ID does not exist"
Jul 17 13:56:24.540786 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:24.540759    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c44494d47-7g9f6"]
Jul 17 13:56:24.546477 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:24.546456    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c44494d47-7g9f6"]
Jul 17 13:56:24.772036 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:24.771941    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80b27150-4ddd-467c-acec-c4fb66953e48" path="/var/lib/kubelet/pods/80b27150-4ddd-467c-acec-c4fb66953e48/volumes"
Jul 17 13:56:34.516581 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:34.516544    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 13:56:38.221382 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.221347    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"]
Jul 17 13:56:38.221807 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.221755    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80b27150-4ddd-467c-acec-c4fb66953e48" containerName="console"
Jul 17 13:56:38.221807 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.221766    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="80b27150-4ddd-467c-acec-c4fb66953e48" containerName="console"
Jul 17 13:56:38.221885 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.221833    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="80b27150-4ddd-467c-acec-c4fb66953e48" containerName="console"
Jul 17 13:56:38.224831 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.224814    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.228479 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.228456    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-jvjj5\""
Jul 17 13:56:38.243873 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.243847    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"]
Jul 17 13:56:38.307732 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.307697    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.307732 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.307732    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr5gp\" (UniqueName: \"kubernetes.io/projected/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-kube-api-access-rr5gp\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.307982 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.307759    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.307982 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.307785    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.307982 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.307810    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.307982 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.307893    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.308237 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.307991    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.308237 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.308081    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.308237 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.308153    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.409527 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.409482    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.409718 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.409554    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.409718 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.409583    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr5gp\" (UniqueName: \"kubernetes.io/projected/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-kube-api-access-rr5gp\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.409718 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.409613    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.409718 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.409641    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.409718 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.409670    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.409718 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.409704    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.410044 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.409760    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.410044 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.409801    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.410733 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.410706    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.410811 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.410789    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.411218 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.411193    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.411456 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.411435    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.411590 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.411524    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.415351 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.413466    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.415351 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.414024    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.421419 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.421391    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.421965 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.421933    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr5gp\" (UniqueName: \"kubernetes.io/projected/3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375-kube-api-access-rr5gp\") pod \"maas-default-gateway-openshift-default-56654dcf7-vvrbt\" (UID: \"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.535371 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.535291    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:38.691100 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.691074    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"]
Jul 17 13:56:38.693521 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:56:38.693485    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bb5bcb8_d6f7_4b9a_8b6c_70f159d2a375.slice/crio-83c68a648e697996a1f9fbaeb87fea895e01b4e36b2cf740b3fe0d79bc3536e5 WatchSource:0}: Error finding container 83c68a648e697996a1f9fbaeb87fea895e01b4e36b2cf740b3fe0d79bc3536e5: Status 404 returned error can't find the container with id 83c68a648e697996a1f9fbaeb87fea895e01b4e36b2cf740b3fe0d79bc3536e5
Jul 17 13:56:38.695831 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.695793    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:56:38.695921 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.695867    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:56:38.695921 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:38.695894    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:56:39.594115 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:39.594070    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt" event={"ID":"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375","Type":"ContainerStarted","Data":"7b9950663e81271bb704340a32861b3416df4410615230f0a737653d7a8bfea7"}
Jul 17 13:56:39.594507 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:39.594141    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt" event={"ID":"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375","Type":"ContainerStarted","Data":"83c68a648e697996a1f9fbaeb87fea895e01b4e36b2cf740b3fe0d79bc3536e5"}
Jul 17 13:56:39.615879 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:39.615824    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt" podStartSLOduration=1.6158050369999999 podStartE2EDuration="1.615805037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:56:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:56:39.615128667 +0000 UTC m=+685.321575883" watchObservedRunningTime="2026-07-17 13:56:39.615805037 +0000 UTC m=+685.322251874"
Jul 17 13:56:40.535719 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:40.535677    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:40.540782 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:40.540751    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:40.598807 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:40.598773    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:40.599799 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:40.599778    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:56:51.351857 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:51.351823    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-5pc6r"]
Jul 17 13:56:51.356760 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:51.356737    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5pc6r"
Jul 17 13:56:51.359480 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:51.359450    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-km8n8\""
Jul 17 13:56:51.363167 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:51.363143    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5pc6r"]
Jul 17 13:56:51.435554 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:51.435515    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phw95\" (UniqueName: \"kubernetes.io/projected/c9f96dc3-3e02-45bd-9a43-3212959d8d08-kube-api-access-phw95\") pod \"authorino-58cd955675-5pc6r\" (UID: \"c9f96dc3-3e02-45bd-9a43-3212959d8d08\") " pod="kuadrant-system/authorino-58cd955675-5pc6r"
Jul 17 13:56:51.536091 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:51.536058    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phw95\" (UniqueName: \"kubernetes.io/projected/c9f96dc3-3e02-45bd-9a43-3212959d8d08-kube-api-access-phw95\") pod \"authorino-58cd955675-5pc6r\" (UID: \"c9f96dc3-3e02-45bd-9a43-3212959d8d08\") " pod="kuadrant-system/authorino-58cd955675-5pc6r"
Jul 17 13:56:51.547336 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:51.547306    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phw95\" (UniqueName: \"kubernetes.io/projected/c9f96dc3-3e02-45bd-9a43-3212959d8d08-kube-api-access-phw95\") pod \"authorino-58cd955675-5pc6r\" (UID: \"c9f96dc3-3e02-45bd-9a43-3212959d8d08\") " pod="kuadrant-system/authorino-58cd955675-5pc6r"
Jul 17 13:56:51.668760 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:51.668736    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5pc6r"
Jul 17 13:56:51.802743 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:51.802625    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5pc6r"]
Jul 17 13:56:51.805462 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:56:51.805432    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9f96dc3_3e02_45bd_9a43_3212959d8d08.slice/crio-e6f7abaf0c40d8cbc336b41b0b3eb2dafd1dcb0443ef96131a0cd8fa5b906c47 WatchSource:0}: Error finding container e6f7abaf0c40d8cbc336b41b0b3eb2dafd1dcb0443ef96131a0cd8fa5b906c47: Status 404 returned error can't find the container with id e6f7abaf0c40d8cbc336b41b0b3eb2dafd1dcb0443ef96131a0cd8fa5b906c47
Jul 17 13:56:52.666801 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:52.666762    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5pc6r" event={"ID":"c9f96dc3-3e02-45bd-9a43-3212959d8d08","Type":"ContainerStarted","Data":"e6f7abaf0c40d8cbc336b41b0b3eb2dafd1dcb0443ef96131a0cd8fa5b906c47"}
Jul 17 13:56:55.688782 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:55.688750    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5pc6r" event={"ID":"c9f96dc3-3e02-45bd-9a43-3212959d8d08","Type":"ContainerStarted","Data":"a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca"}
Jul 17 13:56:55.703705 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:55.703661    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-5pc6r" podStartSLOduration=1.237229218 podStartE2EDuration="4.703646251s" totalImagesPullingTime="3.466417033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:56:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:56:55.702255017 +0000 UTC m=+701.408701852" watchObservedRunningTime="2026-07-17 13:56:55.703646251 +0000 UTC m=+701.410093085"
Jul 17 13:56:56.394833 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:56.394791    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5pc6r"]
Jul 17 13:56:57.699738 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:57.699696    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-5pc6r" podUID="c9f96dc3-3e02-45bd-9a43-3212959d8d08" containerName="authorino" containerID="cri-o://a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca" gracePeriod=30
Jul 17 13:56:57.961499 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:57.961442    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5pc6r"
Jul 17 13:56:58.000415 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.000377    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phw95\" (UniqueName: \"kubernetes.io/projected/c9f96dc3-3e02-45bd-9a43-3212959d8d08-kube-api-access-phw95\") pod \"c9f96dc3-3e02-45bd-9a43-3212959d8d08\" (UID: \"c9f96dc3-3e02-45bd-9a43-3212959d8d08\") "
Jul 17 13:56:58.002649 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.002617    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f96dc3-3e02-45bd-9a43-3212959d8d08-kube-api-access-phw95" (OuterVolumeSpecName: "kube-api-access-phw95") pod "c9f96dc3-3e02-45bd-9a43-3212959d8d08" (UID: "c9f96dc3-3e02-45bd-9a43-3212959d8d08"). InnerVolumeSpecName "kube-api-access-phw95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:56:58.101368 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.101331    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phw95\" (UniqueName: \"kubernetes.io/projected/c9f96dc3-3e02-45bd-9a43-3212959d8d08-kube-api-access-phw95\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:56:58.705985 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.705948    2561 generic.go:356] "Generic (PLEG): container finished" podID="c9f96dc3-3e02-45bd-9a43-3212959d8d08" containerID="a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca" exitCode=0
Jul 17 13:56:58.706470 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.705986    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5pc6r" event={"ID":"c9f96dc3-3e02-45bd-9a43-3212959d8d08","Type":"ContainerDied","Data":"a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca"}
Jul 17 13:56:58.706470 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.706049    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5pc6r"
Jul 17 13:56:58.706470 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.706065    2561 scope.go:117] "RemoveContainer" containerID="a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca"
Jul 17 13:56:58.706470 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.706053    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5pc6r" event={"ID":"c9f96dc3-3e02-45bd-9a43-3212959d8d08","Type":"ContainerDied","Data":"e6f7abaf0c40d8cbc336b41b0b3eb2dafd1dcb0443ef96131a0cd8fa5b906c47"}
Jul 17 13:56:58.715173 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.715155    2561 scope.go:117] "RemoveContainer" containerID="a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca"
Jul 17 13:56:58.715466 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:56:58.715448    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca\": container with ID starting with a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca not found: ID does not exist" containerID="a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca"
Jul 17 13:56:58.715514 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.715476    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca"} err="failed to get container status \"a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca\": rpc error: code = NotFound desc = could not find container \"a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca\": container with ID starting with a50d8722bf59564a009ddd56ef724ac30c4bb1cb62a9081e50d339a4f9ae25ca not found: ID does not exist"
Jul 17 13:56:58.728852 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.728826    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5pc6r"]
Jul 17 13:56:58.732622 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.732601    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-5pc6r"]
Jul 17 13:56:58.772391 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:56:58.772364    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9f96dc3-3e02-45bd-9a43-3212959d8d08" path="/var/lib/kubelet/pods/c9f96dc3-3e02-45bd-9a43-3212959d8d08/volumes"
Jul 17 13:57:20.435135 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.435099    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzm6l"]
Jul 17 13:57:20.435527 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.435500    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9f96dc3-3e02-45bd-9a43-3212959d8d08" containerName="authorino"
Jul 17 13:57:20.435527 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.435513    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9f96dc3-3e02-45bd-9a43-3212959d8d08" containerName="authorino"
Jul 17 13:57:20.435619 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.435609    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9f96dc3-3e02-45bd-9a43-3212959d8d08" containerName="authorino"
Jul 17 13:57:20.439135 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.439119    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xzm6l"
Jul 17 13:57:20.442148 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.442126    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-km8n8\""
Jul 17 13:57:20.447569 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.447542    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzm6l"]
Jul 17 13:57:20.595138 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.595105    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vmml\" (UniqueName: \"kubernetes.io/projected/10cc2b77-e058-4ab2-9448-9e014d733f30-kube-api-access-5vmml\") pod \"authorino-5d4b67c446-xzm6l\" (UID: \"10cc2b77-e058-4ab2-9448-9e014d733f30\") " pod="kuadrant-system/authorino-5d4b67c446-xzm6l"
Jul 17 13:57:20.688334 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.688268    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzm6l"]
Jul 17 13:57:20.688513 ip-10-0-137-163 kubenswrapper[2561]: E0717 13:57:20.688495    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5vmml], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-xzm6l" podUID="10cc2b77-e058-4ab2-9448-9e014d733f30"
Jul 17 13:57:20.696417 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.696394    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vmml\" (UniqueName: \"kubernetes.io/projected/10cc2b77-e058-4ab2-9448-9e014d733f30-kube-api-access-5vmml\") pod \"authorino-5d4b67c446-xzm6l\" (UID: \"10cc2b77-e058-4ab2-9448-9e014d733f30\") " pod="kuadrant-system/authorino-5d4b67c446-xzm6l"
Jul 17 13:57:20.717512 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.717493    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vmml\" (UniqueName: \"kubernetes.io/projected/10cc2b77-e058-4ab2-9448-9e014d733f30-kube-api-access-5vmml\") pod \"authorino-5d4b67c446-xzm6l\" (UID: \"10cc2b77-e058-4ab2-9448-9e014d733f30\") " pod="kuadrant-system/authorino-5d4b67c446-xzm6l"
Jul 17 13:57:20.816508 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.816481    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xzm6l"
Jul 17 13:57:20.822443 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.822423    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xzm6l"
Jul 17 13:57:20.898201 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.898173    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vmml\" (UniqueName: \"kubernetes.io/projected/10cc2b77-e058-4ab2-9448-9e014d733f30-kube-api-access-5vmml\") pod \"10cc2b77-e058-4ab2-9448-9e014d733f30\" (UID: \"10cc2b77-e058-4ab2-9448-9e014d733f30\") "
Jul 17 13:57:20.900394 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.900371    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10cc2b77-e058-4ab2-9448-9e014d733f30-kube-api-access-5vmml" (OuterVolumeSpecName: "kube-api-access-5vmml") pod "10cc2b77-e058-4ab2-9448-9e014d733f30" (UID: "10cc2b77-e058-4ab2-9448-9e014d733f30"). InnerVolumeSpecName "kube-api-access-5vmml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 13:57:20.999557 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:20.999496    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vmml\" (UniqueName: \"kubernetes.io/projected/10cc2b77-e058-4ab2-9448-9e014d733f30-kube-api-access-5vmml\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 13:57:21.100984 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.100953    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85bddb7dc8-fpdlq"]
Jul 17 13:57:21.103346 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.103327    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85bddb7dc8-fpdlq"
Jul 17 13:57:21.107029 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.106995    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 17 13:57:21.114704 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.114684    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85bddb7dc8-fpdlq"]
Jul 17 13:57:21.200675 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.200644    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl6z4\" (UniqueName: \"kubernetes.io/projected/5e881362-797f-4276-8f9b-4e5d780c99c8-kube-api-access-bl6z4\") pod \"authorino-85bddb7dc8-fpdlq\" (UID: \"5e881362-797f-4276-8f9b-4e5d780c99c8\") " pod="kuadrant-system/authorino-85bddb7dc8-fpdlq"
Jul 17 13:57:21.200811 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.200686    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e881362-797f-4276-8f9b-4e5d780c99c8-tls-cert\") pod \"authorino-85bddb7dc8-fpdlq\" (UID: \"5e881362-797f-4276-8f9b-4e5d780c99c8\") " pod="kuadrant-system/authorino-85bddb7dc8-fpdlq"
Jul 17 13:57:21.301815 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.301753    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e881362-797f-4276-8f9b-4e5d780c99c8-tls-cert\") pod \"authorino-85bddb7dc8-fpdlq\" (UID: \"5e881362-797f-4276-8f9b-4e5d780c99c8\") " pod="kuadrant-system/authorino-85bddb7dc8-fpdlq"
Jul 17 13:57:21.301963 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.301864    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl6z4\" (UniqueName: \"kubernetes.io/projected/5e881362-797f-4276-8f9b-4e5d780c99c8-kube-api-access-bl6z4\") pod \"authorino-85bddb7dc8-fpdlq\" (UID: \"5e881362-797f-4276-8f9b-4e5d780c99c8\") " pod="kuadrant-system/authorino-85bddb7dc8-fpdlq"
Jul 17 13:57:21.304373 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.304350    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e881362-797f-4276-8f9b-4e5d780c99c8-tls-cert\") pod \"authorino-85bddb7dc8-fpdlq\" (UID: \"5e881362-797f-4276-8f9b-4e5d780c99c8\") " pod="kuadrant-system/authorino-85bddb7dc8-fpdlq"
Jul 17 13:57:21.312460 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.312436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl6z4\" (UniqueName: \"kubernetes.io/projected/5e881362-797f-4276-8f9b-4e5d780c99c8-kube-api-access-bl6z4\") pod \"authorino-85bddb7dc8-fpdlq\" (UID: \"5e881362-797f-4276-8f9b-4e5d780c99c8\") " pod="kuadrant-system/authorino-85bddb7dc8-fpdlq"
Jul 17 13:57:21.412762 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.412734    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85bddb7dc8-fpdlq"
Jul 17 13:57:21.560197 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.560174    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85bddb7dc8-fpdlq"]
Jul 17 13:57:21.562646 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:57:21.562621    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e881362_797f_4276_8f9b_4e5d780c99c8.slice/crio-18dd5cc2aa2e39da01527a3c1c5732a38840a0a66d2e29c9dfd122ab4e9ed244 WatchSource:0}: Error finding container 18dd5cc2aa2e39da01527a3c1c5732a38840a0a66d2e29c9dfd122ab4e9ed244: Status 404 returned error can't find the container with id 18dd5cc2aa2e39da01527a3c1c5732a38840a0a66d2e29c9dfd122ab4e9ed244
Jul 17 13:57:21.822871 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.822806    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85bddb7dc8-fpdlq" event={"ID":"5e881362-797f-4276-8f9b-4e5d780c99c8","Type":"ContainerStarted","Data":"18dd5cc2aa2e39da01527a3c1c5732a38840a0a66d2e29c9dfd122ab4e9ed244"}
Jul 17 13:57:21.822871 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.822858    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xzm6l"
Jul 17 13:57:21.857449 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.857418    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzm6l"]
Jul 17 13:57:21.859201 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:21.859180    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xzm6l"]
Jul 17 13:57:22.771744 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:22.771704    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10cc2b77-e058-4ab2-9448-9e014d733f30" path="/var/lib/kubelet/pods/10cc2b77-e058-4ab2-9448-9e014d733f30/volumes"
Jul 17 13:57:22.829305 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:22.829273    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85bddb7dc8-fpdlq" event={"ID":"5e881362-797f-4276-8f9b-4e5d780c99c8","Type":"ContainerStarted","Data":"54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7"}
Jul 17 13:57:22.872942 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:57:22.872899    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-85bddb7dc8-fpdlq" podStartSLOduration=1.456582517 podStartE2EDuration="1.872885062s" totalImagesPullingTime="416.302545ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:57:22.869817946 +0000 UTC m=+728.576264780" watchObservedRunningTime="2026-07-17 13:57:22.872885062 +0000 UTC m=+728.579331896"
Jul 17 13:58:11.642162 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:11.642122    2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vvrbt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 13:58:11.642608 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:11.642219    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt" podUID="3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 13:58:13.091406 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:13.091324    2561 generic.go:356] "Generic (PLEG): container finished" podID="3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375" containerID="7b9950663e81271bb704340a32861b3416df4410615230f0a737653d7a8bfea7" exitCode=0
Jul 17 13:58:13.091406 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:13.091393    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt" event={"ID":"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375","Type":"ContainerDied","Data":"7b9950663e81271bb704340a32861b3416df4410615230f0a737653d7a8bfea7"}
Jul 17 13:58:13.091849 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:13.091770    2561 scope.go:117] "RemoveContainer" containerID="7b9950663e81271bb704340a32861b3416df4410615230f0a737653d7a8bfea7"
Jul 17 13:58:13.092748 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:13.092720    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:58:13.092822 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:13.092790    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:58:13.092858 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:13.092821    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 13:58:14.098388 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:14.098344    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt" event={"ID":"3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375","Type":"ContainerStarted","Data":"a3d7be1bae28604e4bcdfea4a29d93cb52c9469204f7688b6a4ab775c6fd3ca4"}
Jul 17 13:58:14.536183 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:14.536144    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:58:14.536496 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:14.536439    2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vvrbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 17 13:58:14.536598 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:14.536528    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt" podUID="3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 17 13:58:15.536574 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:15.536538    2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vvrbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 17 13:58:15.537176 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:15.537143    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt" podUID="3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 17 13:58:16.536067 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:16.536027    2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vvrbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 17 13:58:16.536358 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:16.536329    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt" podUID="3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 17 13:58:17.536367 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:17.536320    2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vvrbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 17 13:58:17.536911 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:17.536400    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt" podUID="3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 17 13:58:18.538700 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:18.538671    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:58:18.539184 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:18.539078    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:58:18.539681 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:18.539667    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vvrbt"
Jul 17 13:58:45.762757 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.762638    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"]
Jul 17 13:58:45.766476 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.766456    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.770060 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.770036    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 17 13:58:45.770060 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.770045    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wn44r\""
Jul 17 13:58:45.770279 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.770080    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 13:58:45.770279 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.770091    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 13:58:45.777669 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.777645    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"]
Jul 17 13:58:45.858282 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.858249    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.858463 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.858321    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.858463 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.858348    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3908cc41-7d8c-4719-b25f-830e1b08a6f1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.858463 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.858386    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.858637 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.858524    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nvp6\" (UniqueName: \"kubernetes.io/projected/3908cc41-7d8c-4719-b25f-830e1b08a6f1-kube-api-access-9nvp6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.858637 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.858563    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.959194 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.959157    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3908cc41-7d8c-4719-b25f-830e1b08a6f1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.959409 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.959218    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.959409 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.959273    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nvp6\" (UniqueName: \"kubernetes.io/projected/3908cc41-7d8c-4719-b25f-830e1b08a6f1-kube-api-access-9nvp6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.959409 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.959300    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.959409 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.959341    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.959409 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.959403    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.959798 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.959767    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.959910 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.959811    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.959910 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.959869    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.962405 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.962377    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3908cc41-7d8c-4719-b25f-830e1b08a6f1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.962555 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.962543    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3908cc41-7d8c-4719-b25f-830e1b08a6f1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:45.968614 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:45.968581    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nvp6\" (UniqueName: \"kubernetes.io/projected/3908cc41-7d8c-4719-b25f-830e1b08a6f1-kube-api-access-9nvp6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vr2gm\" (UID: \"3908cc41-7d8c-4719-b25f-830e1b08a6f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:46.078644 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:46.078541    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:46.218102 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:46.218066    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"]
Jul 17 13:58:46.219955 ip-10-0-137-163 kubenswrapper[2561]: W0717 13:58:46.219929    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3908cc41_7d8c_4719_b25f_830e1b08a6f1.slice/crio-b12c9bf9742ce7d5f85e6c211377cf93f245f7055a229edb30d16bcd38c50781 WatchSource:0}: Error finding container b12c9bf9742ce7d5f85e6c211377cf93f245f7055a229edb30d16bcd38c50781: Status 404 returned error can't find the container with id b12c9bf9742ce7d5f85e6c211377cf93f245f7055a229edb30d16bcd38c50781
Jul 17 13:58:46.262116 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:46.262074    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm" event={"ID":"3908cc41-7d8c-4719-b25f-830e1b08a6f1","Type":"ContainerStarted","Data":"b12c9bf9742ce7d5f85e6c211377cf93f245f7055a229edb30d16bcd38c50781"}
Jul 17 13:58:48.275212 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:48.275168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm" event={"ID":"3908cc41-7d8c-4719-b25f-830e1b08a6f1","Type":"ContainerStarted","Data":"d8b8b7d251cca717907afd1760cdcbd149cc873e09c5f9dafa1e2ff0785bab52"}
Jul 17 13:58:48.292432 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:48.292378    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm" podStartSLOduration=1.860392038 podStartE2EDuration="3.292362352s" totalImagesPullingTime="1.431970314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:58:48.291782427 +0000 UTC m=+813.998229264" watchObservedRunningTime="2026-07-17 13:58:48.292362352 +0000 UTC m=+813.998809186"
Jul 17 13:58:56.078917 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:56.078885    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:56.079418 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:56.078947    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:56.091377 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:56.091360    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 13:58:56.326938 ip-10-0-137-163 kubenswrapper[2561]: I0717 13:58:56.326909    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vr2gm"
Jul 17 14:00:00.148982 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:00.148946    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k"]
Jul 17 14:00:00.152664 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:00.152649    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k"
Jul 17 14:00:00.155489 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:00.155470    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:00:00.159758 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:00.159733    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k"]
Jul 17 14:00:00.187207 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:00.187189    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k"
Jul 17 14:00:00.534776 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:00.534753    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k"]
Jul 17 14:00:00.537144 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:00:00.537117    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9088857f_996d_4183_8e8c_db56fdf45e8e.slice/crio-e43c708287a2cb90f7c828775bed89b31171cfe800bd586a1d727e2c3ca5ab5a WatchSource:0}: Error finding container e43c708287a2cb90f7c828775bed89b31171cfe800bd586a1d727e2c3ca5ab5a: Status 404 returned error can't find the container with id e43c708287a2cb90f7c828775bed89b31171cfe800bd586a1d727e2c3ca5ab5a
Jul 17 14:00:00.657469 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:00.657440    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k" event={"ID":"9088857f-996d-4183-8e8c-db56fdf45e8e","Type":"ContainerStarted","Data":"e43c708287a2cb90f7c828775bed89b31171cfe800bd586a1d727e2c3ca5ab5a"}
Jul 17 14:00:04.680809 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:04.680773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k" event={"ID":"9088857f-996d-4183-8e8c-db56fdf45e8e","Type":"ContainerStarted","Data":"7cc0647446089ddb4dd2900ae261d9248824dcbfada1067918205d1fdbde4df1"}
Jul 17 14:00:04.696566 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:04.696514    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k" podStartSLOduration=1.020194355 podStartE2EDuration="4.696495504s" totalImagesPullingTime="3.676301149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:00:04.694906634 +0000 UTC m=+890.401353467" watchObservedRunningTime="2026-07-17 14:00:04.696495504 +0000 UTC m=+890.402942334"
Jul 17 14:00:05.687744 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:05.687668    2561 generic.go:356] "Generic (PLEG): container finished" podID="9088857f-996d-4183-8e8c-db56fdf45e8e" containerID="7cc0647446089ddb4dd2900ae261d9248824dcbfada1067918205d1fdbde4df1" exitCode=0
Jul 17 14:00:05.688110 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:05.687759    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k" event={"ID":"9088857f-996d-4183-8e8c-db56fdf45e8e","Type":"ContainerDied","Data":"7cc0647446089ddb4dd2900ae261d9248824dcbfada1067918205d1fdbde4df1"}
Jul 17 14:00:06.841104 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:06.841082    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k"
Jul 17 14:00:07.699432 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:07.699407    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k"
Jul 17 14:00:07.699432 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:07.699426    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k" event={"ID":"9088857f-996d-4183-8e8c-db56fdf45e8e","Type":"ContainerDied","Data":"e43c708287a2cb90f7c828775bed89b31171cfe800bd586a1d727e2c3ca5ab5a"}
Jul 17 14:00:07.699635 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:07.699466    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e43c708287a2cb90f7c828775bed89b31171cfe800bd586a1d727e2c3ca5ab5a"
Jul 17 14:00:19.599611 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.599578    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-fdbc7f8c9-xljlk"]
Jul 17 14:00:19.600245 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.600162    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9088857f-996d-4183-8e8c-db56fdf45e8e" containerName="cleanup"
Jul 17 14:00:19.600245 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.600184    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9088857f-996d-4183-8e8c-db56fdf45e8e" containerName="cleanup"
Jul 17 14:00:19.600347 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.600310    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9088857f-996d-4183-8e8c-db56fdf45e8e" containerName="cleanup"
Jul 17 14:00:19.603506 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.603490    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fdbc7f8c9-xljlk"
Jul 17 14:00:19.609883 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.609856    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fdbc7f8c9-xljlk"]
Jul 17 14:00:19.730161 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.730127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43dc360c-549c-4d9b-8dbf-24ca9941c60b-tls-cert\") pod \"authorino-fdbc7f8c9-xljlk\" (UID: \"43dc360c-549c-4d9b-8dbf-24ca9941c60b\") " pod="kuadrant-system/authorino-fdbc7f8c9-xljlk"
Jul 17 14:00:19.730322 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.730169    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbp26\" (UniqueName: \"kubernetes.io/projected/43dc360c-549c-4d9b-8dbf-24ca9941c60b-kube-api-access-sbp26\") pod \"authorino-fdbc7f8c9-xljlk\" (UID: \"43dc360c-549c-4d9b-8dbf-24ca9941c60b\") " pod="kuadrant-system/authorino-fdbc7f8c9-xljlk"
Jul 17 14:00:19.830872 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.830840    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43dc360c-549c-4d9b-8dbf-24ca9941c60b-tls-cert\") pod \"authorino-fdbc7f8c9-xljlk\" (UID: \"43dc360c-549c-4d9b-8dbf-24ca9941c60b\") " pod="kuadrant-system/authorino-fdbc7f8c9-xljlk"
Jul 17 14:00:19.831051 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.830888    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbp26\" (UniqueName: \"kubernetes.io/projected/43dc360c-549c-4d9b-8dbf-24ca9941c60b-kube-api-access-sbp26\") pod \"authorino-fdbc7f8c9-xljlk\" (UID: \"43dc360c-549c-4d9b-8dbf-24ca9941c60b\") " pod="kuadrant-system/authorino-fdbc7f8c9-xljlk"
Jul 17 14:00:19.833492 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.833461    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43dc360c-549c-4d9b-8dbf-24ca9941c60b-tls-cert\") pod \"authorino-fdbc7f8c9-xljlk\" (UID: \"43dc360c-549c-4d9b-8dbf-24ca9941c60b\") " pod="kuadrant-system/authorino-fdbc7f8c9-xljlk"
Jul 17 14:00:19.838743 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.838718    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbp26\" (UniqueName: \"kubernetes.io/projected/43dc360c-549c-4d9b-8dbf-24ca9941c60b-kube-api-access-sbp26\") pod \"authorino-fdbc7f8c9-xljlk\" (UID: \"43dc360c-549c-4d9b-8dbf-24ca9941c60b\") " pod="kuadrant-system/authorino-fdbc7f8c9-xljlk"
Jul 17 14:00:19.914149 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:19.914078    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fdbc7f8c9-xljlk"
Jul 17 14:00:20.052326 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:20.052298    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fdbc7f8c9-xljlk"]
Jul 17 14:00:20.054220 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:00:20.054193    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43dc360c_549c_4d9b_8dbf_24ca9941c60b.slice/crio-cc64686fe81b7205071ad7ccd34cc4532ff145214b3e4875367d90c3d4a9e802 WatchSource:0}: Error finding container cc64686fe81b7205071ad7ccd34cc4532ff145214b3e4875367d90c3d4a9e802: Status 404 returned error can't find the container with id cc64686fe81b7205071ad7ccd34cc4532ff145214b3e4875367d90c3d4a9e802
Jul 17 14:00:20.775029 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:20.774924    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fdbc7f8c9-xljlk" event={"ID":"43dc360c-549c-4d9b-8dbf-24ca9941c60b","Type":"ContainerStarted","Data":"b1ffb971c88e5cd37697432187f27e2f7023f4d84e052a8aa7ba4443bff54304"}
Jul 17 14:00:20.775029 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:20.774970    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fdbc7f8c9-xljlk" event={"ID":"43dc360c-549c-4d9b-8dbf-24ca9941c60b","Type":"ContainerStarted","Data":"cc64686fe81b7205071ad7ccd34cc4532ff145214b3e4875367d90c3d4a9e802"}
Jul 17 14:00:20.795695 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:20.795632    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-fdbc7f8c9-xljlk" podStartSLOduration=1.347467957 podStartE2EDuration="1.795612802s" totalImagesPullingTime="448.144845ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:00:20.792434994 +0000 UTC m=+906.498881832" watchObservedRunningTime="2026-07-17 14:00:20.795612802 +0000 UTC m=+906.502059638"
Jul 17 14:00:20.827695 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:20.827662    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85bddb7dc8-fpdlq"]
Jul 17 14:00:20.828439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:20.828150    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-85bddb7dc8-fpdlq" podUID="5e881362-797f-4276-8f9b-4e5d780c99c8" containerName="authorino" containerID="cri-o://54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7" gracePeriod=30
Jul 17 14:00:21.108788 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.108762    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85bddb7dc8-fpdlq"
Jul 17 14:00:21.244139 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.244103    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl6z4\" (UniqueName: \"kubernetes.io/projected/5e881362-797f-4276-8f9b-4e5d780c99c8-kube-api-access-bl6z4\") pod \"5e881362-797f-4276-8f9b-4e5d780c99c8\" (UID: \"5e881362-797f-4276-8f9b-4e5d780c99c8\") "
Jul 17 14:00:21.244315 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.244186    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e881362-797f-4276-8f9b-4e5d780c99c8-tls-cert\") pod \"5e881362-797f-4276-8f9b-4e5d780c99c8\" (UID: \"5e881362-797f-4276-8f9b-4e5d780c99c8\") "
Jul 17 14:00:21.246469 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.246406    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e881362-797f-4276-8f9b-4e5d780c99c8-kube-api-access-bl6z4" (OuterVolumeSpecName: "kube-api-access-bl6z4") pod "5e881362-797f-4276-8f9b-4e5d780c99c8" (UID: "5e881362-797f-4276-8f9b-4e5d780c99c8"). InnerVolumeSpecName "kube-api-access-bl6z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:00:21.255799 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.255773    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e881362-797f-4276-8f9b-4e5d780c99c8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5e881362-797f-4276-8f9b-4e5d780c99c8" (UID: "5e881362-797f-4276-8f9b-4e5d780c99c8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 14:00:21.345668 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.345580    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl6z4\" (UniqueName: \"kubernetes.io/projected/5e881362-797f-4276-8f9b-4e5d780c99c8-kube-api-access-bl6z4\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:00:21.345668 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.345614    2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e881362-797f-4276-8f9b-4e5d780c99c8-tls-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:00:21.780902 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.780862    2561 generic.go:356] "Generic (PLEG): container finished" podID="5e881362-797f-4276-8f9b-4e5d780c99c8" containerID="54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7" exitCode=0
Jul 17 14:00:21.781418 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.781227    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85bddb7dc8-fpdlq"
Jul 17 14:00:21.781512 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.781482    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85bddb7dc8-fpdlq" event={"ID":"5e881362-797f-4276-8f9b-4e5d780c99c8","Type":"ContainerDied","Data":"54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7"}
Jul 17 14:00:21.781599 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.781519    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85bddb7dc8-fpdlq" event={"ID":"5e881362-797f-4276-8f9b-4e5d780c99c8","Type":"ContainerDied","Data":"18dd5cc2aa2e39da01527a3c1c5732a38840a0a66d2e29c9dfd122ab4e9ed244"}
Jul 17 14:00:21.781599 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.781541    2561 scope.go:117] "RemoveContainer" containerID="54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7"
Jul 17 14:00:21.792727 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.792707    2561 scope.go:117] "RemoveContainer" containerID="54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7"
Jul 17 14:00:21.793059 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:00:21.793030    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7\": container with ID starting with 54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7 not found: ID does not exist" containerID="54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7"
Jul 17 14:00:21.793161 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.793070    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7"} err="failed to get container status \"54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7\": rpc error: code = NotFound desc = could not find container \"54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7\": container with ID starting with 54c6d97ddf9b3cf5a6eca4ae63f8853c83f25553ee06298ae374160720e6d5d7 not found: ID does not exist"
Jul 17 14:00:21.803166 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.803143    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85bddb7dc8-fpdlq"]
Jul 17 14:00:21.809715 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:21.809689    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-85bddb7dc8-fpdlq"]
Jul 17 14:00:22.773263 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:00:22.773232    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e881362-797f-4276-8f9b-4e5d780c99c8" path="/var/lib/kubelet/pods/5e881362-797f-4276-8f9b-4e5d780c99c8/volumes"
Jul 17 14:07:10.261971 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:10.261933    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"]
Jul 17 14:07:10.262479 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:10.262248    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf" podUID="60875591-4314-475a-9258-4bd9bba42999" containerName="manager" containerID="cri-o://8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887" gracePeriod=10
Jul 17 14:07:12.921999 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:12.921976    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 14:07:12.937129 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:12.937105    2561 generic.go:356] "Generic (PLEG): container finished" podID="60875591-4314-475a-9258-4bd9bba42999" containerID="8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887" exitCode=0
Jul 17 14:07:12.937246 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:12.937142    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"
Jul 17 14:07:12.937246 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:12.937186    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf" event={"ID":"60875591-4314-475a-9258-4bd9bba42999","Type":"ContainerDied","Data":"8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887"}
Jul 17 14:07:12.937246 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:12.937240    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf" event={"ID":"60875591-4314-475a-9258-4bd9bba42999","Type":"ContainerDied","Data":"d0e3f83844fdbe322bd7e9185e8fd15084dfa69a1f2a4191528bd978016eaeca"}
Jul 17 14:07:12.937374 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:12.937267    2561 scope.go:117] "RemoveContainer" containerID="8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887"
Jul 17 14:07:12.946838 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:12.946822    2561 scope.go:117] "RemoveContainer" containerID="8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887"
Jul 17 14:07:12.947151 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:07:12.947131    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887\": container with ID starting with 8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887 not found: ID does not exist" containerID="8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887"
Jul 17 14:07:12.947232 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:12.947163    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887"} err="failed to get container status \"8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887\": rpc error: code = NotFound desc = could not find container \"8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887\": container with ID starting with 8600f0d016fc11017c0f3d8d30596d9d59ff421216cfd6fb8a30de1882894887 not found: ID does not exist"
Jul 17 14:07:13.002447 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:13.002364    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7kcq\" (UniqueName: \"kubernetes.io/projected/60875591-4314-475a-9258-4bd9bba42999-kube-api-access-k7kcq\") pod \"60875591-4314-475a-9258-4bd9bba42999\" (UID: \"60875591-4314-475a-9258-4bd9bba42999\") "
Jul 17 14:07:13.002581 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:13.002464    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60875591-4314-475a-9258-4bd9bba42999-extensions-socket-volume\") pod \"60875591-4314-475a-9258-4bd9bba42999\" (UID: \"60875591-4314-475a-9258-4bd9bba42999\") "
Jul 17 14:07:13.002836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:13.002815    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60875591-4314-475a-9258-4bd9bba42999-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "60875591-4314-475a-9258-4bd9bba42999" (UID: "60875591-4314-475a-9258-4bd9bba42999"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:07:13.004617 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:13.004587    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60875591-4314-475a-9258-4bd9bba42999-kube-api-access-k7kcq" (OuterVolumeSpecName: "kube-api-access-k7kcq") pod "60875591-4314-475a-9258-4bd9bba42999" (UID: "60875591-4314-475a-9258-4bd9bba42999"). InnerVolumeSpecName "kube-api-access-k7kcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:07:13.104070 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:13.104043    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60875591-4314-475a-9258-4bd9bba42999-extensions-socket-volume\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:07:13.104070 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:13.104066    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7kcq\" (UniqueName: \"kubernetes.io/projected/60875591-4314-475a-9258-4bd9bba42999-kube-api-access-k7kcq\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:07:13.263353 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:13.263281    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"]
Jul 17 14:07:13.265523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:13.265502    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xtvmf"]
Jul 17 14:07:14.772566 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:07:14.772534    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60875591-4314-475a-9258-4bd9bba42999" path="/var/lib/kubelet/pods/60875591-4314-475a-9258-4bd9bba42999/volumes"
Jul 17 14:08:11.588998 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.588963    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"]
Jul 17 14:08:11.589460 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.589390    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60875591-4314-475a-9258-4bd9bba42999" containerName="manager"
Jul 17 14:08:11.589460 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.589402    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="60875591-4314-475a-9258-4bd9bba42999" containerName="manager"
Jul 17 14:08:11.589460 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.589410    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e881362-797f-4276-8f9b-4e5d780c99c8" containerName="authorino"
Jul 17 14:08:11.589460 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.589414    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e881362-797f-4276-8f9b-4e5d780c99c8" containerName="authorino"
Jul 17 14:08:11.589590 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.589472    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="60875591-4314-475a-9258-4bd9bba42999" containerName="manager"
Jul 17 14:08:11.589590 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.589484    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e881362-797f-4276-8f9b-4e5d780c99c8" containerName="authorino"
Jul 17 14:08:11.592579 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.592563    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"
Jul 17 14:08:11.595552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.595530    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bzpxn\""
Jul 17 14:08:11.604477 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.604458    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"]
Jul 17 14:08:11.737374 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.737347    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s762\" (UniqueName: \"kubernetes.io/projected/bb136535-9959-49b8-a70e-45ff92f81ef3-kube-api-access-6s762\") pod \"kuadrant-operator-controller-manager-86878c769f-j4qjr\" (UID: \"bb136535-9959-49b8-a70e-45ff92f81ef3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"
Jul 17 14:08:11.737515 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.737386    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb136535-9959-49b8-a70e-45ff92f81ef3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j4qjr\" (UID: \"bb136535-9959-49b8-a70e-45ff92f81ef3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"
Jul 17 14:08:11.838143 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.838120    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s762\" (UniqueName: \"kubernetes.io/projected/bb136535-9959-49b8-a70e-45ff92f81ef3-kube-api-access-6s762\") pod \"kuadrant-operator-controller-manager-86878c769f-j4qjr\" (UID: \"bb136535-9959-49b8-a70e-45ff92f81ef3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"
Jul 17 14:08:11.838248 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.838158    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb136535-9959-49b8-a70e-45ff92f81ef3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j4qjr\" (UID: \"bb136535-9959-49b8-a70e-45ff92f81ef3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"
Jul 17 14:08:11.838562 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.838545    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb136535-9959-49b8-a70e-45ff92f81ef3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j4qjr\" (UID: \"bb136535-9959-49b8-a70e-45ff92f81ef3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"
Jul 17 14:08:11.856203 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.856147    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s762\" (UniqueName: \"kubernetes.io/projected/bb136535-9959-49b8-a70e-45ff92f81ef3-kube-api-access-6s762\") pod \"kuadrant-operator-controller-manager-86878c769f-j4qjr\" (UID: \"bb136535-9959-49b8-a70e-45ff92f81ef3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"
Jul 17 14:08:11.903113 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:11.903081    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"
Jul 17 14:08:12.051062 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:12.051028    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"]
Jul 17 14:08:12.053225 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:08:12.053197    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb136535_9959_49b8_a70e_45ff92f81ef3.slice/crio-4b8ca238fb88733a0f7e002813cdc99ef3ce2999466684efa7ac4e8c296f4044 WatchSource:0}: Error finding container 4b8ca238fb88733a0f7e002813cdc99ef3ce2999466684efa7ac4e8c296f4044: Status 404 returned error can't find the container with id 4b8ca238fb88733a0f7e002813cdc99ef3ce2999466684efa7ac4e8c296f4044
Jul 17 14:08:12.243255 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:12.243218    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr" event={"ID":"bb136535-9959-49b8-a70e-45ff92f81ef3","Type":"ContainerStarted","Data":"d0a2000cd16024a5fec836390ede680373f36477ca4d95e18d060ca2237327a8"}
Jul 17 14:08:12.243441 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:12.243301    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"
Jul 17 14:08:12.243441 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:12.243322    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr" event={"ID":"bb136535-9959-49b8-a70e-45ff92f81ef3","Type":"ContainerStarted","Data":"4b8ca238fb88733a0f7e002813cdc99ef3ce2999466684efa7ac4e8c296f4044"}
Jul 17 14:08:12.270304 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:12.270260    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr" podStartSLOduration=1.270246586 podStartE2EDuration="1.270246586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:08:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:08:12.268131981 +0000 UTC m=+1377.974578820" watchObservedRunningTime="2026-07-17 14:08:12.270246586 +0000 UTC m=+1377.976693420"
Jul 17 14:08:23.250974 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:08:23.250944    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j4qjr"
Jul 17 14:14:16.200201 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.200117    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"]
Jul 17 14:14:16.204437 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.204413    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.207898 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.207872    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-988d1f69-openshift-default-dockercfg-kwfzl\""
Jul 17 14:14:16.228385 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.228352    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"]
Jul 17 14:14:16.274834 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.274802    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-envoy\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.275064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.274845    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-data\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.275064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.274874    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-socket\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.275064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.274923    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-token\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.275064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.274942    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-certs\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.275064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.275033    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-podinfo\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.275336 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.275091    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-credential-socket\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.275336 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.275180    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istiod-ca-cert\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.275336 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.275210    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwnld\" (UniqueName: \"kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-kube-api-access-hwnld\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.375735 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.375700    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istiod-ca-cert\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.375735 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.375738    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwnld\" (UniqueName: \"kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-kube-api-access-hwnld\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.375997 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.375770    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-envoy\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.375997 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.375808    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-data\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.375997 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.375850    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-socket\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.375997 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.375882    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-token\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.376312 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.376060    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-certs\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.376312 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.376133    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-podinfo\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.376312 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.376182    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-credential-socket\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.376312 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.376302    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-socket\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.376497 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.376379    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-data\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.376497 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.376470    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-certs\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.376578 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.376515    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-credential-socket\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.376614 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.376593    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istiod-ca-cert\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.378360 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.378335    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-envoy\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.378580 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.378563    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-podinfo\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.384964 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.384939    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-token\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.385095 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.385052    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwnld\" (UniqueName: \"kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-kube-api-access-hwnld\") pod \"e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") " pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.516486 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.516392    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:16.667499 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.667459    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"]
Jul 17 14:14:16.669185 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:14:16.669140    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf307dfd6_72f3_48ce_98f0_bcfaaed390ae.slice/crio-e95e33e3a7cd3d87eb0a21895389c5cd10bc0d09f18412bcc7416fd27163b30b WatchSource:0}: Error finding container e95e33e3a7cd3d87eb0a21895389c5cd10bc0d09f18412bcc7416fd27163b30b: Status 404 returned error can't find the container with id e95e33e3a7cd3d87eb0a21895389c5cd10bc0d09f18412bcc7416fd27163b30b
Jul 17 14:14:16.671463 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.671431    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:14:16.671553 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.671500    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:14:16.671553 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:16.671526    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:14:17.220353 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:17.220313    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" event={"ID":"f307dfd6-72f3-48ce-98f0-bcfaaed390ae","Type":"ContainerStarted","Data":"7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4"}
Jul 17 14:14:17.220760 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:17.220403    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" event={"ID":"f307dfd6-72f3-48ce-98f0-bcfaaed390ae","Type":"ContainerStarted","Data":"e95e33e3a7cd3d87eb0a21895389c5cd10bc0d09f18412bcc7416fd27163b30b"}
Jul 17 14:14:17.249314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:17.249265    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" podStartSLOduration=1.249246941 podStartE2EDuration="1.249246941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:14:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:14:17.24838329 +0000 UTC m=+1742.954830124" watchObservedRunningTime="2026-07-17 14:14:17.249246941 +0000 UTC m=+1742.955693774"
Jul 17 14:14:17.517192 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:17.517096    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:17.522126 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:17.522100    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:18.226616 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:18.226577    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:18.227697 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:18.227675    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:22.432120 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:22.432077    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"]
Jul 17 14:14:22.437493 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:22.437470    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"
Jul 17 14:14:22.443052 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:22.443024    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:14:22.443301 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:22.443281    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:14:22.449909 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:22.449877    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"]
Jul 17 14:14:22.541031 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:22.539530    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d80ec8b-8553-4739-93a2-0a7432fe9835-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-ds45r\" (UID: \"3d80ec8b-8553-4739-93a2-0a7432fe9835\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"
Jul 17 14:14:22.640767 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:22.640721    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d80ec8b-8553-4739-93a2-0a7432fe9835-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-ds45r\" (UID: \"3d80ec8b-8553-4739-93a2-0a7432fe9835\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"
Jul 17 14:14:22.641661 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:22.641629    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d80ec8b-8553-4739-93a2-0a7432fe9835-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-ds45r\" (UID: \"3d80ec8b-8553-4739-93a2-0a7432fe9835\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"
Jul 17 14:14:22.758276 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:22.758202    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"
Jul 17 14:14:23.132042 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:23.131920    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"]
Jul 17 14:14:23.137873 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:14:23.137832    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d80ec8b_8553_4739_93a2_0a7432fe9835.slice/crio-444913b3ae276bdb714d7fe7ebcc39d34ec02ec144e2935f1e0015b46a2b0e99 WatchSource:0}: Error finding container 444913b3ae276bdb714d7fe7ebcc39d34ec02ec144e2935f1e0015b46a2b0e99: Status 404 returned error can't find the container with id 444913b3ae276bdb714d7fe7ebcc39d34ec02ec144e2935f1e0015b46a2b0e99
Jul 17 14:14:23.259194 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:23.259158    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r" event={"ID":"3d80ec8b-8553-4739-93a2-0a7432fe9835","Type":"ContainerStarted","Data":"444913b3ae276bdb714d7fe7ebcc39d34ec02ec144e2935f1e0015b46a2b0e99"}
Jul 17 14:14:24.266670 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:24.266635    2561 generic.go:356] "Generic (PLEG): container finished" podID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerID="b19757ac2055fde5629848a9109017eee573849e12689b620b909976f8d06fbd" exitCode=7
Jul 17 14:14:24.266670 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:24.266673    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r" event={"ID":"3d80ec8b-8553-4739-93a2-0a7432fe9835","Type":"ContainerDied","Data":"b19757ac2055fde5629848a9109017eee573849e12689b620b909976f8d06fbd"}
Jul 17 14:14:24.267208 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:24.267088    2561 scope.go:117] "RemoveContainer" containerID="b19757ac2055fde5629848a9109017eee573849e12689b620b909976f8d06fbd"
Jul 17 14:14:25.273436 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:25.273356    2561 generic.go:356] "Generic (PLEG): container finished" podID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerID="48b7386b2f795c375f10aae675c583e43e6034d5bdebf4d3caeedc7ac622697a" exitCode=7
Jul 17 14:14:25.273857 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:25.273441    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r" event={"ID":"3d80ec8b-8553-4739-93a2-0a7432fe9835","Type":"ContainerDied","Data":"48b7386b2f795c375f10aae675c583e43e6034d5bdebf4d3caeedc7ac622697a"}
Jul 17 14:14:25.273857 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:25.273506    2561 scope.go:117] "RemoveContainer" containerID="b19757ac2055fde5629848a9109017eee573849e12689b620b909976f8d06fbd"
Jul 17 14:14:25.273857 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:25.273688    2561 scope.go:117] "RemoveContainer" containerID="48b7386b2f795c375f10aae675c583e43e6034d5bdebf4d3caeedc7ac622697a"
Jul 17 14:14:25.274053 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:14:25.273915    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-ds45r_odh-ai-gateway-infra(3d80ec8b-8553-4739-93a2-0a7432fe9835)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835"
Jul 17 14:14:26.282437 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:26.282411    2561 scope.go:117] "RemoveContainer" containerID="48b7386b2f795c375f10aae675c583e43e6034d5bdebf4d3caeedc7ac622697a"
Jul 17 14:14:26.282833 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:14:26.282628    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-ds45r_odh-ai-gateway-infra(3d80ec8b-8553-4739-93a2-0a7432fe9835)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835"
Jul 17 14:14:28.294924 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:28.294841    2561 generic.go:356] "Generic (PLEG): container finished" podID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerID="7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4" exitCode=0
Jul 17 14:14:28.295451 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:28.294924    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" event={"ID":"f307dfd6-72f3-48ce-98f0-bcfaaed390ae","Type":"ContainerDied","Data":"7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4"}
Jul 17 14:14:28.295518 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:28.295496    2561 scope.go:117] "RemoveContainer" containerID="7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4"
Jul 17 14:14:28.296535 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:28.296506    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:14:28.296640 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:28.296600    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:14:28.296704 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:28.296655    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:14:29.302682 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:29.302646    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" event={"ID":"f307dfd6-72f3-48ce-98f0-bcfaaed390ae","Type":"ContainerStarted","Data":"963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992"}
Jul 17 14:14:29.517312 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:29.517277    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:29.518463 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:29.518441    2561 patch_prober.go:28] interesting pod/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 17 14:14:29.518522 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:29.518493    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 17 14:14:30.517078 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:30.517044    2561 patch_prober.go:28] interesting pod/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 17 14:14:30.517454 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:30.517094    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 17 14:14:31.517717 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:31.517678    2561 patch_prober.go:28] interesting pod/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 17 14:14:31.518121 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:31.517742    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 17 14:14:32.517089 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:32.517053    2561 patch_prober.go:28] interesting pod/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 17 14:14:32.517271 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:32.517105    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 17 14:14:33.226730 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:33.226686    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:33.521591 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:33.521510    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:33.522523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:33.522502    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:14:39.765783 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:39.765751    2561 scope.go:117] "RemoveContainer" containerID="48b7386b2f795c375f10aae675c583e43e6034d5bdebf4d3caeedc7ac622697a"
Jul 17 14:14:40.370527 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:40.370496    2561 generic.go:356] "Generic (PLEG): container finished" podID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerID="240cfaaf6c904ac64b01ec9caacd901e778d30fe0825a28f866e23f63c20267d" exitCode=7
Jul 17 14:14:40.370670 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:40.370542    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r" event={"ID":"3d80ec8b-8553-4739-93a2-0a7432fe9835","Type":"ContainerDied","Data":"240cfaaf6c904ac64b01ec9caacd901e778d30fe0825a28f866e23f63c20267d"}
Jul 17 14:14:40.370670 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:40.370609    2561 scope.go:117] "RemoveContainer" containerID="48b7386b2f795c375f10aae675c583e43e6034d5bdebf4d3caeedc7ac622697a"
Jul 17 14:14:40.370919 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:40.370895    2561 scope.go:117] "RemoveContainer" containerID="240cfaaf6c904ac64b01ec9caacd901e778d30fe0825a28f866e23f63c20267d"
Jul 17 14:14:40.371127 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:14:40.371104    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-ds45r_odh-ai-gateway-infra(3d80ec8b-8553-4739-93a2-0a7432fe9835)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835"
Jul 17 14:14:41.399190 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:41.399162    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"]
Jul 17 14:14:41.540408 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:41.540387    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"
Jul 17 14:14:41.729782 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:41.729759    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d80ec8b-8553-4739-93a2-0a7432fe9835-maas-api-service-ca\") pod \"3d80ec8b-8553-4739-93a2-0a7432fe9835\" (UID: \"3d80ec8b-8553-4739-93a2-0a7432fe9835\") "
Jul 17 14:14:41.730114 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:41.730092    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d80ec8b-8553-4739-93a2-0a7432fe9835-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3d80ec8b-8553-4739-93a2-0a7432fe9835" (UID: "3d80ec8b-8553-4739-93a2-0a7432fe9835"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:14:41.830867 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:41.830840    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d80ec8b-8553-4739-93a2-0a7432fe9835-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:14:42.383046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:42.382997    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"
Jul 17 14:14:42.383046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:42.383030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r" event={"ID":"3d80ec8b-8553-4739-93a2-0a7432fe9835","Type":"ContainerDied","Data":"444913b3ae276bdb714d7fe7ebcc39d34ec02ec144e2935f1e0015b46a2b0e99"}
Jul 17 14:14:42.383279 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:42.383093    2561 scope.go:117] "RemoveContainer" containerID="240cfaaf6c904ac64b01ec9caacd901e778d30fe0825a28f866e23f63c20267d"
Jul 17 14:14:42.406484 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:42.406460    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"]
Jul 17 14:14:42.416690 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:42.416668    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-ds45r"]
Jul 17 14:14:42.772202 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:14:42.772169    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835" path="/var/lib/kubelet/pods/3d80ec8b-8553-4739-93a2-0a7432fe9835/volumes"
Jul 17 14:15:00.138061 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.138024    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738295-jqt5q"]
Jul 17 14:15:00.138883 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.138863    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerName="revoke-keys"
Jul 17 14:15:00.138956 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.138888    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerName="revoke-keys"
Jul 17 14:15:00.138956 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.138909    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerName="revoke-keys"
Jul 17 14:15:00.138956 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.138919    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerName="revoke-keys"
Jul 17 14:15:00.139090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.139028    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerName="revoke-keys"
Jul 17 14:15:00.139090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.139041    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerName="revoke-keys"
Jul 17 14:15:00.142078 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.142055    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w"]
Jul 17 14:15:00.142179 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.142163    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738295-jqt5q"
Jul 17 14:15:00.142540 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.142524    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerName="revoke-keys"
Jul 17 14:15:00.142581 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.142544    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerName="revoke-keys"
Jul 17 14:15:00.142650 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.142639    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d80ec8b-8553-4739-93a2-0a7432fe9835" containerName="revoke-keys"
Jul 17 14:15:00.144766 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.144740    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:15:00.145584 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.145565    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w"
Jul 17 14:15:00.149157 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.149131    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738295-jqt5q"]
Jul 17 14:15:00.151755 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.151726    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w"]
Jul 17 14:15:00.155817 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.155796    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738295-jqt5q"
Jul 17 14:15:00.164138 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.164119    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w"
Jul 17 14:15:00.305404 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.305364    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738295-jqt5q"]
Jul 17 14:15:00.306701 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:15:00.306667    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda38e9089_f0b7_4713_a760_41f11750263f.slice/crio-d943a60e209787ee53542ba43e99817359f4e39eb6e9d0a2d3ad2876dced5ded WatchSource:0}: Error finding container d943a60e209787ee53542ba43e99817359f4e39eb6e9d0a2d3ad2876dced5ded: Status 404 returned error can't find the container with id d943a60e209787ee53542ba43e99817359f4e39eb6e9d0a2d3ad2876dced5ded
Jul 17 14:15:00.333760 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.333723    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w"]
Jul 17 14:15:00.336143 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:15:00.336118    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd73e7c_283b_4662_977c_436227b13706.slice/crio-2e7989c3b86e59e63770b5b595f3d35d4e1e7a62d00e468e9c506adcbf36c40e WatchSource:0}: Error finding container 2e7989c3b86e59e63770b5b595f3d35d4e1e7a62d00e468e9c506adcbf36c40e: Status 404 returned error can't find the container with id 2e7989c3b86e59e63770b5b595f3d35d4e1e7a62d00e468e9c506adcbf36c40e
Jul 17 14:15:00.489369 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.489339    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w" event={"ID":"fdd73e7c-283b-4662-977c-436227b13706","Type":"ContainerStarted","Data":"2e7989c3b86e59e63770b5b595f3d35d4e1e7a62d00e468e9c506adcbf36c40e"}
Jul 17 14:15:00.490468 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:00.490446    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738295-jqt5q" event={"ID":"a38e9089-f0b7-4713-a760-41f11750263f","Type":"ContainerStarted","Data":"d943a60e209787ee53542ba43e99817359f4e39eb6e9d0a2d3ad2876dced5ded"}
Jul 17 14:15:01.497034 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:01.496987    2561 generic.go:356] "Generic (PLEG): container finished" podID="a38e9089-f0b7-4713-a760-41f11750263f" containerID="91c17ad9030b4d0c5fac9d13becdc740a45f454c1386fb2048a2df7d36c3b577" exitCode=0
Jul 17 14:15:01.497481 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:01.497082    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738295-jqt5q" event={"ID":"a38e9089-f0b7-4713-a760-41f11750263f","Type":"ContainerDied","Data":"91c17ad9030b4d0c5fac9d13becdc740a45f454c1386fb2048a2df7d36c3b577"}
Jul 17 14:15:01.498671 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:01.498647    2561 generic.go:356] "Generic (PLEG): container finished" podID="fdd73e7c-283b-4662-977c-436227b13706" containerID="ca4fe3c525213bde80cb9653c3c3dfc9c1f0ab86db6e2f49642b8d103044811d" exitCode=7
Jul 17 14:15:01.498793 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:01.498703    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w" event={"ID":"fdd73e7c-283b-4662-977c-436227b13706","Type":"ContainerDied","Data":"ca4fe3c525213bde80cb9653c3c3dfc9c1f0ab86db6e2f49642b8d103044811d"}
Jul 17 14:15:01.498866 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:01.498854    2561 scope.go:117] "RemoveContainer" containerID="ca4fe3c525213bde80cb9653c3c3dfc9c1f0ab86db6e2f49642b8d103044811d"
Jul 17 14:15:02.506128 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:02.506044    2561 generic.go:356] "Generic (PLEG): container finished" podID="fdd73e7c-283b-4662-977c-436227b13706" containerID="a5112d01aa09fca5ef330c188c016d77553e76d2b101df03ffe6b28a069714c2" exitCode=7
Jul 17 14:15:02.506128 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:02.506095    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w" event={"ID":"fdd73e7c-283b-4662-977c-436227b13706","Type":"ContainerDied","Data":"a5112d01aa09fca5ef330c188c016d77553e76d2b101df03ffe6b28a069714c2"}
Jul 17 14:15:02.506564 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:02.506157    2561 scope.go:117] "RemoveContainer" containerID="ca4fe3c525213bde80cb9653c3c3dfc9c1f0ab86db6e2f49642b8d103044811d"
Jul 17 14:15:02.506564 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:02.506277    2561 scope.go:117] "RemoveContainer" containerID="a5112d01aa09fca5ef330c188c016d77553e76d2b101df03ffe6b28a069714c2"
Jul 17 14:15:02.506564 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:15:02.506526    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w_odh-ai-gateway-infra(fdd73e7c-283b-4662-977c-436227b13706)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w" podUID="fdd73e7c-283b-4662-977c-436227b13706"
Jul 17 14:15:02.649093 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:02.649064    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738295-jqt5q"
Jul 17 14:15:03.513449 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:03.513420    2561 scope.go:117] "RemoveContainer" containerID="a5112d01aa09fca5ef330c188c016d77553e76d2b101df03ffe6b28a069714c2"
Jul 17 14:15:03.513875 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:15:03.513630    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w_odh-ai-gateway-infra(fdd73e7c-283b-4662-977c-436227b13706)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w" podUID="fdd73e7c-283b-4662-977c-436227b13706"
Jul 17 14:15:03.514585 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:03.514570    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738295-jqt5q"
Jul 17 14:15:03.514658 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:03.514607    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738295-jqt5q" event={"ID":"a38e9089-f0b7-4713-a760-41f11750263f","Type":"ContainerDied","Data":"d943a60e209787ee53542ba43e99817359f4e39eb6e9d0a2d3ad2876dced5ded"}
Jul 17 14:15:03.514658 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:03.514643    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d943a60e209787ee53542ba43e99817359f4e39eb6e9d0a2d3ad2876dced5ded"
Jul 17 14:15:14.768768 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:14.768734    2561 scope.go:117] "RemoveContainer" containerID="a5112d01aa09fca5ef330c188c016d77553e76d2b101df03ffe6b28a069714c2"
Jul 17 14:15:15.581173 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:15.581146    2561 generic.go:356] "Generic (PLEG): container finished" podID="fdd73e7c-283b-4662-977c-436227b13706" containerID="19849c72fe2deb9116ebda095c4dd409ff4c17cc4b566a738d628759edb7e09e" exitCode=7
Jul 17 14:15:15.581289 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:15.581224    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w" event={"ID":"fdd73e7c-283b-4662-977c-436227b13706","Type":"ContainerDied","Data":"19849c72fe2deb9116ebda095c4dd409ff4c17cc4b566a738d628759edb7e09e"}
Jul 17 14:15:15.581289 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:15.581284    2561 scope.go:117] "RemoveContainer" containerID="a5112d01aa09fca5ef330c188c016d77553e76d2b101df03ffe6b28a069714c2"
Jul 17 14:15:15.581502 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:15.581484    2561 scope.go:117] "RemoveContainer" containerID="19849c72fe2deb9116ebda095c4dd409ff4c17cc4b566a738d628759edb7e09e"
Jul 17 14:15:15.581761 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:15:15.581742    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w_odh-ai-gateway-infra(fdd73e7c-283b-4662-977c-436227b13706)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w" podUID="fdd73e7c-283b-4662-977c-436227b13706"
Jul 17 14:15:16.606877 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:16.606842    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w"]
Jul 17 14:15:16.742297 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:16.742276    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w"
Jul 17 14:15:17.594272 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:17.594234    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w" event={"ID":"fdd73e7c-283b-4662-977c-436227b13706","Type":"ContainerDied","Data":"2e7989c3b86e59e63770b5b595f3d35d4e1e7a62d00e468e9c506adcbf36c40e"}
Jul 17 14:15:17.594272 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:17.594260    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w"
Jul 17 14:15:17.594496 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:17.594307    2561 scope.go:117] "RemoveContainer" containerID="19849c72fe2deb9116ebda095c4dd409ff4c17cc4b566a738d628759edb7e09e"
Jul 17 14:15:17.611206 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:17.611182    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w"]
Jul 17 14:15:17.616597 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:17.616577    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-988d1f69-29738295-pws4w"]
Jul 17 14:15:18.771780 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:18.771746    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdd73e7c-283b-4662-977c-436227b13706" path="/var/lib/kubelet/pods/fdd73e7c-283b-4662-977c-436227b13706/volumes"
Jul 17 14:15:22.502880 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.502845    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"]
Jul 17 14:15:22.503356 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.503340    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdd73e7c-283b-4662-977c-436227b13706" containerName="cleanup"
Jul 17 14:15:22.503398 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.503358    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdd73e7c-283b-4662-977c-436227b13706" containerName="cleanup"
Jul 17 14:15:22.503398 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.503372    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdd73e7c-283b-4662-977c-436227b13706" containerName="cleanup"
Jul 17 14:15:22.503398 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.503386    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdd73e7c-283b-4662-977c-436227b13706" containerName="cleanup"
Jul 17 14:15:22.503398 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.503397    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a38e9089-f0b7-4713-a760-41f11750263f" containerName="cleanup"
Jul 17 14:15:22.503523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.503403    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a38e9089-f0b7-4713-a760-41f11750263f" containerName="cleanup"
Jul 17 14:15:22.503523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.503479    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdd73e7c-283b-4662-977c-436227b13706" containerName="cleanup"
Jul 17 14:15:22.503523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.503488    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a38e9089-f0b7-4713-a760-41f11750263f" containerName="cleanup"
Jul 17 14:15:22.503523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.503494    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdd73e7c-283b-4662-977c-436227b13706" containerName="cleanup"
Jul 17 14:15:22.507858 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.507840    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"
Jul 17 14:15:22.511360 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.511342    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:15:22.511444 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.511361    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:15:22.517763 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.517737    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"]
Jul 17 14:15:22.682635 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.682608    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c068fac4-3a4b-464d-8e83-bd139c6fc55a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc\" (UID: \"c068fac4-3a4b-464d-8e83-bd139c6fc55a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"
Jul 17 14:15:22.783328 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.783265    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c068fac4-3a4b-464d-8e83-bd139c6fc55a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc\" (UID: \"c068fac4-3a4b-464d-8e83-bd139c6fc55a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"
Jul 17 14:15:22.783832 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.783813    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c068fac4-3a4b-464d-8e83-bd139c6fc55a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc\" (UID: \"c068fac4-3a4b-464d-8e83-bd139c6fc55a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"
Jul 17 14:15:22.819294 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.819273    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"
Jul 17 14:15:22.954917 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:22.954893    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"]
Jul 17 14:15:22.957410 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:15:22.957369    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc068fac4_3a4b_464d_8e83_bd139c6fc55a.slice/crio-9d407718f3b184171a670cd3e6d60d53cf7e9fda2eda48d7eb629e100f8e8c8b WatchSource:0}: Error finding container 9d407718f3b184171a670cd3e6d60d53cf7e9fda2eda48d7eb629e100f8e8c8b: Status 404 returned error can't find the container with id 9d407718f3b184171a670cd3e6d60d53cf7e9fda2eda48d7eb629e100f8e8c8b
Jul 17 14:15:23.629852 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:23.629819    2561 generic.go:356] "Generic (PLEG): container finished" podID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerID="fcbea83612599a1454bcfe8d328a69769995c10d8895af31de0b3dc92aff4d2f" exitCode=7
Jul 17 14:15:23.630251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:23.629913    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc" event={"ID":"c068fac4-3a4b-464d-8e83-bd139c6fc55a","Type":"ContainerDied","Data":"fcbea83612599a1454bcfe8d328a69769995c10d8895af31de0b3dc92aff4d2f"}
Jul 17 14:15:23.630251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:23.629983    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc" event={"ID":"c068fac4-3a4b-464d-8e83-bd139c6fc55a","Type":"ContainerStarted","Data":"9d407718f3b184171a670cd3e6d60d53cf7e9fda2eda48d7eb629e100f8e8c8b"}
Jul 17 14:15:23.630251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:23.630098    2561 scope.go:117] "RemoveContainer" containerID="fcbea83612599a1454bcfe8d328a69769995c10d8895af31de0b3dc92aff4d2f"
Jul 17 14:15:24.637282 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:24.637197    2561 generic.go:356] "Generic (PLEG): container finished" podID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerID="b5c2441cb1cb53ad9c52ca537e1aa7140c6eefb7474dd5c20bac5f71ded8fc3d" exitCode=7
Jul 17 14:15:24.637671 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:24.637287    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc" event={"ID":"c068fac4-3a4b-464d-8e83-bd139c6fc55a","Type":"ContainerDied","Data":"b5c2441cb1cb53ad9c52ca537e1aa7140c6eefb7474dd5c20bac5f71ded8fc3d"}
Jul 17 14:15:24.637671 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:24.637344    2561 scope.go:117] "RemoveContainer" containerID="fcbea83612599a1454bcfe8d328a69769995c10d8895af31de0b3dc92aff4d2f"
Jul 17 14:15:24.637671 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:24.637475    2561 scope.go:117] "RemoveContainer" containerID="b5c2441cb1cb53ad9c52ca537e1aa7140c6eefb7474dd5c20bac5f71ded8fc3d"
Jul 17 14:15:24.637834 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:15:24.637722    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc_odh-ai-gateway-infra(c068fac4-3a4b-464d-8e83-bd139c6fc55a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a"
Jul 17 14:15:25.644669 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:25.644636    2561 scope.go:117] "RemoveContainer" containerID="b5c2441cb1cb53ad9c52ca537e1aa7140c6eefb7474dd5c20bac5f71ded8fc3d"
Jul 17 14:15:25.645053 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:15:25.644821    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc_odh-ai-gateway-infra(c068fac4-3a4b-464d-8e83-bd139c6fc55a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a"
Jul 17 14:15:40.766375 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:40.766345    2561 scope.go:117] "RemoveContainer" containerID="b5c2441cb1cb53ad9c52ca537e1aa7140c6eefb7474dd5c20bac5f71ded8fc3d"
Jul 17 14:15:41.731819 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:41.731786    2561 generic.go:356] "Generic (PLEG): container finished" podID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerID="eb976a59eddfbd9dc57753004bf1c6525ed0eeb099786a0881e405ab855de21b" exitCode=7
Jul 17 14:15:41.732022 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:41.731860    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc" event={"ID":"c068fac4-3a4b-464d-8e83-bd139c6fc55a","Type":"ContainerDied","Data":"eb976a59eddfbd9dc57753004bf1c6525ed0eeb099786a0881e405ab855de21b"}
Jul 17 14:15:41.732022 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:41.731920    2561 scope.go:117] "RemoveContainer" containerID="b5c2441cb1cb53ad9c52ca537e1aa7140c6eefb7474dd5c20bac5f71ded8fc3d"
Jul 17 14:15:41.732232 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:41.732210    2561 scope.go:117] "RemoveContainer" containerID="eb976a59eddfbd9dc57753004bf1c6525ed0eeb099786a0881e405ab855de21b"
Jul 17 14:15:41.732421 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:15:41.732398    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc_odh-ai-gateway-infra(c068fac4-3a4b-464d-8e83-bd139c6fc55a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a"
Jul 17 14:15:42.759411 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:42.759371    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"]
Jul 17 14:15:42.912429 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:42.912406    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"
Jul 17 14:15:42.980731 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:42.980705    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c068fac4-3a4b-464d-8e83-bd139c6fc55a-maas-api-service-ca\") pod \"c068fac4-3a4b-464d-8e83-bd139c6fc55a\" (UID: \"c068fac4-3a4b-464d-8e83-bd139c6fc55a\") "
Jul 17 14:15:42.981044 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:42.981021    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c068fac4-3a4b-464d-8e83-bd139c6fc55a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c068fac4-3a4b-464d-8e83-bd139c6fc55a" (UID: "c068fac4-3a4b-464d-8e83-bd139c6fc55a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:15:43.082380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:43.082317    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c068fac4-3a4b-464d-8e83-bd139c6fc55a-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:15:43.744852 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:43.744823    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc" event={"ID":"c068fac4-3a4b-464d-8e83-bd139c6fc55a","Type":"ContainerDied","Data":"9d407718f3b184171a670cd3e6d60d53cf7e9fda2eda48d7eb629e100f8e8c8b"}
Jul 17 14:15:43.744852 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:43.744851    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"
Jul 17 14:15:43.745102 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:43.744879    2561 scope.go:117] "RemoveContainer" containerID="eb976a59eddfbd9dc57753004bf1c6525ed0eeb099786a0881e405ab855de21b"
Jul 17 14:15:43.769873 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:43.769842    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"]
Jul 17 14:15:43.775444 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:43.775422    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-9vqgc"]
Jul 17 14:15:44.771588 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:15:44.771554    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" path="/var/lib/kubelet/pods/c068fac4-3a4b-464d-8e83-bd139c6fc55a/volumes"
Jul 17 14:16:22.530478 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530444    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"]
Jul 17 14:16:22.530877 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530856    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdd73e7c-283b-4662-977c-436227b13706" containerName="cleanup"
Jul 17 14:16:22.530877 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530867    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdd73e7c-283b-4662-977c-436227b13706" containerName="cleanup"
Jul 17 14:16:22.530954 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530881    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerName="revoke-keys"
Jul 17 14:16:22.530954 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530886    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerName="revoke-keys"
Jul 17 14:16:22.530954 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530897    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerName="revoke-keys"
Jul 17 14:16:22.530954 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530903    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerName="revoke-keys"
Jul 17 14:16:22.531102 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530974    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerName="revoke-keys"
Jul 17 14:16:22.531102 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530985    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerName="revoke-keys"
Jul 17 14:16:22.531102 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530991    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerName="revoke-keys"
Jul 17 14:16:22.531102 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.530998    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdd73e7c-283b-4662-977c-436227b13706" containerName="cleanup"
Jul 17 14:16:22.533984 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.533970    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"
Jul 17 14:16:22.536535 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.536517    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:16:22.536624 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.536523    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:16:22.542861 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.542839    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"]
Jul 17 14:16:22.629557 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.629529    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9715fa58-6a83-4404-b969-94281e33a004-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z\" (UID: \"9715fa58-6a83-4404-b969-94281e33a004\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"
Jul 17 14:16:22.730455 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.730428    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9715fa58-6a83-4404-b969-94281e33a004-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z\" (UID: \"9715fa58-6a83-4404-b969-94281e33a004\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"
Jul 17 14:16:22.730986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.730970    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9715fa58-6a83-4404-b969-94281e33a004-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z\" (UID: \"9715fa58-6a83-4404-b969-94281e33a004\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"
Jul 17 14:16:22.845396 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.845338    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"
Jul 17 14:16:22.983229 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:22.983205    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"]
Jul 17 14:16:22.984449 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:16:22.984421    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9715fa58_6a83_4404_b969_94281e33a004.slice/crio-526b805ff04282a378a536f128e991ae347be941249bdcc3bbac30621b5eb775 WatchSource:0}: Error finding container 526b805ff04282a378a536f128e991ae347be941249bdcc3bbac30621b5eb775: Status 404 returned error can't find the container with id 526b805ff04282a378a536f128e991ae347be941249bdcc3bbac30621b5eb775
Jul 17 14:16:23.954830 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:23.954793    2561 generic.go:356] "Generic (PLEG): container finished" podID="9715fa58-6a83-4404-b969-94281e33a004" containerID="f8d6205ec9b0fa0cb998c83c851b5e564410ef29fea419371a699c18e39d0290" exitCode=7
Jul 17 14:16:23.955286 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:23.954847    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z" event={"ID":"9715fa58-6a83-4404-b969-94281e33a004","Type":"ContainerDied","Data":"f8d6205ec9b0fa0cb998c83c851b5e564410ef29fea419371a699c18e39d0290"}
Jul 17 14:16:23.955286 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:23.954917    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z" event={"ID":"9715fa58-6a83-4404-b969-94281e33a004","Type":"ContainerStarted","Data":"526b805ff04282a378a536f128e991ae347be941249bdcc3bbac30621b5eb775"}
Jul 17 14:16:23.955286 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:23.955137    2561 scope.go:117] "RemoveContainer" containerID="f8d6205ec9b0fa0cb998c83c851b5e564410ef29fea419371a699c18e39d0290"
Jul 17 14:16:24.962091 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:24.962050    2561 generic.go:356] "Generic (PLEG): container finished" podID="9715fa58-6a83-4404-b969-94281e33a004" containerID="fb316da009280c73b933e3e432d1e30d4c956cb3ad69ead53cbf755eb7718d32" exitCode=7
Jul 17 14:16:24.962558 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:24.962147    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z" event={"ID":"9715fa58-6a83-4404-b969-94281e33a004","Type":"ContainerDied","Data":"fb316da009280c73b933e3e432d1e30d4c956cb3ad69ead53cbf755eb7718d32"}
Jul 17 14:16:24.962558 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:24.962199    2561 scope.go:117] "RemoveContainer" containerID="f8d6205ec9b0fa0cb998c83c851b5e564410ef29fea419371a699c18e39d0290"
Jul 17 14:16:24.962558 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:24.962417    2561 scope.go:117] "RemoveContainer" containerID="fb316da009280c73b933e3e432d1e30d4c956cb3ad69ead53cbf755eb7718d32"
Jul 17 14:16:24.962727 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:16:24.962658    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z_odh-ai-gateway-infra(9715fa58-6a83-4404-b969-94281e33a004)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z" podUID="9715fa58-6a83-4404-b969-94281e33a004"
Jul 17 14:16:25.968802 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:25.968777    2561 scope.go:117] "RemoveContainer" containerID="fb316da009280c73b933e3e432d1e30d4c956cb3ad69ead53cbf755eb7718d32"
Jul 17 14:16:25.969208 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:16:25.969046    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z_odh-ai-gateway-infra(9715fa58-6a83-4404-b969-94281e33a004)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z" podUID="9715fa58-6a83-4404-b969-94281e33a004"
Jul 17 14:16:37.766574 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:37.766493    2561 scope.go:117] "RemoveContainer" containerID="fb316da009280c73b933e3e432d1e30d4c956cb3ad69ead53cbf755eb7718d32"
Jul 17 14:16:39.046486 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:39.046456    2561 generic.go:356] "Generic (PLEG): container finished" podID="9715fa58-6a83-4404-b969-94281e33a004" containerID="c46107ee79367497a1f553154e4ffe18e75ae1ddc4fd5287fba3336ea7e5c021" exitCode=7
Jul 17 14:16:39.046895 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:39.046531    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z" event={"ID":"9715fa58-6a83-4404-b969-94281e33a004","Type":"ContainerDied","Data":"c46107ee79367497a1f553154e4ffe18e75ae1ddc4fd5287fba3336ea7e5c021"}
Jul 17 14:16:39.046895 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:39.046589    2561 scope.go:117] "RemoveContainer" containerID="fb316da009280c73b933e3e432d1e30d4c956cb3ad69ead53cbf755eb7718d32"
Jul 17 14:16:39.046895 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:39.046833    2561 scope.go:117] "RemoveContainer" containerID="c46107ee79367497a1f553154e4ffe18e75ae1ddc4fd5287fba3336ea7e5c021"
Jul 17 14:16:39.047072 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:16:39.047052    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z_odh-ai-gateway-infra(9715fa58-6a83-4404-b969-94281e33a004)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z" podUID="9715fa58-6a83-4404-b969-94281e33a004"
Jul 17 14:16:40.077939 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:40.077907    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"]
Jul 17 14:16:40.226496 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:40.226473    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"
Jul 17 14:16:40.300723 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:40.300694    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9715fa58-6a83-4404-b969-94281e33a004-maas-api-service-ca\") pod \"9715fa58-6a83-4404-b969-94281e33a004\" (UID: \"9715fa58-6a83-4404-b969-94281e33a004\") "
Jul 17 14:16:40.301042 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:40.300995    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9715fa58-6a83-4404-b969-94281e33a004-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9715fa58-6a83-4404-b969-94281e33a004" (UID: "9715fa58-6a83-4404-b969-94281e33a004"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:16:40.402357 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:40.402292    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9715fa58-6a83-4404-b969-94281e33a004-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:16:41.060052 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:41.059941    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z" event={"ID":"9715fa58-6a83-4404-b969-94281e33a004","Type":"ContainerDied","Data":"526b805ff04282a378a536f128e991ae347be941249bdcc3bbac30621b5eb775"}
Jul 17 14:16:41.060052 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:41.060027    2561 scope.go:117] "RemoveContainer" containerID="c46107ee79367497a1f553154e4ffe18e75ae1ddc4fd5287fba3336ea7e5c021"
Jul 17 14:16:41.060288 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:41.059959    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"
Jul 17 14:16:41.080314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:41.080289    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"]
Jul 17 14:16:41.087178 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:41.087159    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fwz6z"]
Jul 17 14:16:42.772188 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:16:42.772153    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9715fa58-6a83-4404-b969-94281e33a004" path="/var/lib/kubelet/pods/9715fa58-6a83-4404-b969-94281e33a004/volumes"
Jul 17 14:17:12.569091 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569055    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"]
Jul 17 14:17:12.569552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569497    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerName="revoke-keys"
Jul 17 14:17:12.569552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569508    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c068fac4-3a4b-464d-8e83-bd139c6fc55a" containerName="revoke-keys"
Jul 17 14:17:12.569552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569544    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9715fa58-6a83-4404-b969-94281e33a004" containerName="revoke-keys"
Jul 17 14:17:12.569552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569551    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9715fa58-6a83-4404-b969-94281e33a004" containerName="revoke-keys"
Jul 17 14:17:12.569686 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569562    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9715fa58-6a83-4404-b969-94281e33a004" containerName="revoke-keys"
Jul 17 14:17:12.569686 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569567    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9715fa58-6a83-4404-b969-94281e33a004" containerName="revoke-keys"
Jul 17 14:17:12.569686 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569574    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9715fa58-6a83-4404-b969-94281e33a004" containerName="revoke-keys"
Jul 17 14:17:12.569686 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569579    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9715fa58-6a83-4404-b969-94281e33a004" containerName="revoke-keys"
Jul 17 14:17:12.569686 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569653    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9715fa58-6a83-4404-b969-94281e33a004" containerName="revoke-keys"
Jul 17 14:17:12.569686 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.569664    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9715fa58-6a83-4404-b969-94281e33a004" containerName="revoke-keys"
Jul 17 14:17:12.573014 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.572986    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"
Jul 17 14:17:12.575401 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.575376    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:17:12.575401 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.575394    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:17:12.580445 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.580421    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"]
Jul 17 14:17:12.670083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.670044    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90b9b65d-2971-44b0-b6ba-a40921ccc64b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-64ldl\" (UID: \"90b9b65d-2971-44b0-b6ba-a40921ccc64b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"
Jul 17 14:17:12.770911 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.770881    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90b9b65d-2971-44b0-b6ba-a40921ccc64b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-64ldl\" (UID: \"90b9b65d-2971-44b0-b6ba-a40921ccc64b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"
Jul 17 14:17:12.771545 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.771528    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90b9b65d-2971-44b0-b6ba-a40921ccc64b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-64ldl\" (UID: \"90b9b65d-2971-44b0-b6ba-a40921ccc64b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"
Jul 17 14:17:12.884957 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:12.884885    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"
Jul 17 14:17:13.016658 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:13.016605    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"]
Jul 17 14:17:13.019075 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:17:13.019048    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b9b65d_2971_44b0_b6ba_a40921ccc64b.slice/crio-b8f15eb6eeded5d55d0a93f73c4ff26c7a2176321e0a839fe1c809263440620a WatchSource:0}: Error finding container b8f15eb6eeded5d55d0a93f73c4ff26c7a2176321e0a839fe1c809263440620a: Status 404 returned error can't find the container with id b8f15eb6eeded5d55d0a93f73c4ff26c7a2176321e0a839fe1c809263440620a
Jul 17 14:17:13.239587 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:13.239557    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl" event={"ID":"90b9b65d-2971-44b0-b6ba-a40921ccc64b","Type":"ContainerStarted","Data":"b8f15eb6eeded5d55d0a93f73c4ff26c7a2176321e0a839fe1c809263440620a"}
Jul 17 14:17:14.246479 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:14.246445    2561 generic.go:356] "Generic (PLEG): container finished" podID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerID="edfbc3129dd33c7cd4750c5d11914b53eb1296824df837872dd530ff6dcdebb4" exitCode=7
Jul 17 14:17:14.246918 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:14.246526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl" event={"ID":"90b9b65d-2971-44b0-b6ba-a40921ccc64b","Type":"ContainerDied","Data":"edfbc3129dd33c7cd4750c5d11914b53eb1296824df837872dd530ff6dcdebb4"}
Jul 17 14:17:14.246918 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:14.246678    2561 scope.go:117] "RemoveContainer" containerID="edfbc3129dd33c7cd4750c5d11914b53eb1296824df837872dd530ff6dcdebb4"
Jul 17 14:17:15.252990 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:15.252959    2561 generic.go:356] "Generic (PLEG): container finished" podID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerID="dcedaa642c66c0caed11041dc781f790b1745958f010bd345c97ca18b185f162" exitCode=7
Jul 17 14:17:15.253460 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:15.253046    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl" event={"ID":"90b9b65d-2971-44b0-b6ba-a40921ccc64b","Type":"ContainerDied","Data":"dcedaa642c66c0caed11041dc781f790b1745958f010bd345c97ca18b185f162"}
Jul 17 14:17:15.253460 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:15.253105    2561 scope.go:117] "RemoveContainer" containerID="edfbc3129dd33c7cd4750c5d11914b53eb1296824df837872dd530ff6dcdebb4"
Jul 17 14:17:15.253460 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:15.253324    2561 scope.go:117] "RemoveContainer" containerID="dcedaa642c66c0caed11041dc781f790b1745958f010bd345c97ca18b185f162"
Jul 17 14:17:15.253629 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:17:15.253574    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-64ldl_odh-ai-gateway-infra(90b9b65d-2971-44b0-b6ba-a40921ccc64b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b"
Jul 17 14:17:16.260102 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:16.260077    2561 scope.go:117] "RemoveContainer" containerID="dcedaa642c66c0caed11041dc781f790b1745958f010bd345c97ca18b185f162"
Jul 17 14:17:16.260507 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:17:16.260279    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-64ldl_odh-ai-gateway-infra(90b9b65d-2971-44b0-b6ba-a40921ccc64b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b"
Jul 17 14:17:22.514404 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:22.514372    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"]
Jul 17 14:17:22.514893 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:22.514575    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy" containerID="cri-o://963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992" gracePeriod=30
Jul 17 14:17:26.766016 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:26.765967    2561 scope.go:117] "RemoveContainer" containerID="dcedaa642c66c0caed11041dc781f790b1745958f010bd345c97ca18b185f162"
Jul 17 14:17:27.321133 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.321099    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl" event={"ID":"90b9b65d-2971-44b0-b6ba-a40921ccc64b","Type":"ContainerStarted","Data":"4a33a8ac8d37f7c10dc432fdb4766cec1173f89ce1887d1c67c74854ec43e47f"}
Jul 17 14:17:27.344899 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.344843    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl" podStartSLOduration=15.344828031 podStartE2EDuration="15.344828031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:17:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:17:27.343182133 +0000 UTC m=+1933.049628997" watchObservedRunningTime="2026-07-17 14:17:27.344828031 +0000 UTC m=+1933.051274865"
Jul 17 14:17:27.774865 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.774835    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:17:27.908706 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.908678    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-socket\") pod \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") "
Jul 17 14:17:27.908848 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.908761    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-token\") pod \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") "
Jul 17 14:17:27.908848 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.908792    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-envoy\") pod \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") "
Jul 17 14:17:27.908932 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.908853    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-certs\") pod \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") "
Jul 17 14:17:27.908932 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.908877    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-credential-socket\") pod \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") "
Jul 17 14:17:27.908932 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.908906    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwnld\" (UniqueName: \"kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-kube-api-access-hwnld\") pod \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") "
Jul 17 14:17:27.909115 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.908945    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-podinfo\") pod \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") "
Jul 17 14:17:27.909115 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.909061    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-data\") pod \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") "
Jul 17 14:17:27.909219 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.909129    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istiod-ca-cert\") pod \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\" (UID: \"f307dfd6-72f3-48ce-98f0-bcfaaed390ae\") "
Jul 17 14:17:27.909219 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.908966    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f307dfd6-72f3-48ce-98f0-bcfaaed390ae" (UID: "f307dfd6-72f3-48ce-98f0-bcfaaed390ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:17:27.909219 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.909194    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f307dfd6-72f3-48ce-98f0-bcfaaed390ae" (UID: "f307dfd6-72f3-48ce-98f0-bcfaaed390ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:17:27.909379 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.909246    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f307dfd6-72f3-48ce-98f0-bcfaaed390ae" (UID: "f307dfd6-72f3-48ce-98f0-bcfaaed390ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:17:27.909536 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.909513    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f307dfd6-72f3-48ce-98f0-bcfaaed390ae" (UID: "f307dfd6-72f3-48ce-98f0-bcfaaed390ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:17:27.910104 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.910079    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "f307dfd6-72f3-48ce-98f0-bcfaaed390ae" (UID: "f307dfd6-72f3-48ce-98f0-bcfaaed390ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:17:27.910245 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.910218    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:17:27.910361 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.910346    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:17:27.910469 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.910445    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:17:27.910583 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.910471    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:17:27.911236 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.911203    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f307dfd6-72f3-48ce-98f0-bcfaaed390ae" (UID: "f307dfd6-72f3-48ce-98f0-bcfaaed390ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:17:27.911321 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.911291    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "f307dfd6-72f3-48ce-98f0-bcfaaed390ae" (UID: "f307dfd6-72f3-48ce-98f0-bcfaaed390ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:17:27.911664 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.911636    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-kube-api-access-hwnld" (OuterVolumeSpecName: "kube-api-access-hwnld") pod "f307dfd6-72f3-48ce-98f0-bcfaaed390ae" (UID: "f307dfd6-72f3-48ce-98f0-bcfaaed390ae"). InnerVolumeSpecName "kube-api-access-hwnld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:17:27.911664 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:27.911645    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f307dfd6-72f3-48ce-98f0-bcfaaed390ae" (UID: "f307dfd6-72f3-48ce-98f0-bcfaaed390ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:17:28.010975 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.010948    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:17:28.010975 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.010973    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:17:28.011141 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.010983    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwnld\" (UniqueName: \"kubernetes.io/projected/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-kube-api-access-hwnld\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:17:28.011141 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.010992    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:17:28.011141 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.011001    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f307dfd6-72f3-48ce-98f0-bcfaaed390ae-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:17:28.330484 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.330450    2561 generic.go:356] "Generic (PLEG): container finished" podID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerID="963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992" exitCode=0
Jul 17 14:17:28.330662 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.330511    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"
Jul 17 14:17:28.330662 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.330540    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" event={"ID":"f307dfd6-72f3-48ce-98f0-bcfaaed390ae","Type":"ContainerDied","Data":"963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992"}
Jul 17 14:17:28.330662 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.330637    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5" event={"ID":"f307dfd6-72f3-48ce-98f0-bcfaaed390ae","Type":"ContainerDied","Data":"e95e33e3a7cd3d87eb0a21895389c5cd10bc0d09f18412bcc7416fd27163b30b"}
Jul 17 14:17:28.330802 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.330669    2561 scope.go:117] "RemoveContainer" containerID="963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992"
Jul 17 14:17:28.332642 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.332616    2561 generic.go:356] "Generic (PLEG): container finished" podID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerID="4a33a8ac8d37f7c10dc432fdb4766cec1173f89ce1887d1c67c74854ec43e47f" exitCode=7
Jul 17 14:17:28.332766 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.332697    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl" event={"ID":"90b9b65d-2971-44b0-b6ba-a40921ccc64b","Type":"ContainerDied","Data":"4a33a8ac8d37f7c10dc432fdb4766cec1173f89ce1887d1c67c74854ec43e47f"}
Jul 17 14:17:28.333066 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.333046    2561 scope.go:117] "RemoveContainer" containerID="4a33a8ac8d37f7c10dc432fdb4766cec1173f89ce1887d1c67c74854ec43e47f"
Jul 17 14:17:28.333342 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:17:28.333317    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-64ldl_odh-ai-gateway-infra(90b9b65d-2971-44b0-b6ba-a40921ccc64b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b"
Jul 17 14:17:28.341109 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.341088    2561 scope.go:117] "RemoveContainer" containerID="7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4"
Jul 17 14:17:28.350360 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.350035    2561 scope.go:117] "RemoveContainer" containerID="963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992"
Jul 17 14:17:28.350430 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:17:28.350404    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992\": container with ID starting with 963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992 not found: ID does not exist" containerID="963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992"
Jul 17 14:17:28.350477 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.350427    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992"} err="failed to get container status \"963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992\": rpc error: code = NotFound desc = could not find container \"963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992\": container with ID starting with 963c545d3426f493bd8cee930e97575e5580da2a8fd2611a0df6515f46e0f992 not found: ID does not exist"
Jul 17 14:17:28.350477 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.350442    2561 scope.go:117] "RemoveContainer" containerID="7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4"
Jul 17 14:17:28.350706 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:17:28.350683    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4\": container with ID starting with 7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4 not found: ID does not exist" containerID="7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4"
Jul 17 14:17:28.350773 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.350711    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4"} err="failed to get container status \"7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4\": rpc error: code = NotFound desc = could not find container \"7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4\": container with ID starting with 7c93486779a855e7f64ced5574269b04660ea0d6019e868103d0b077899893f4 not found: ID does not exist"
Jul 17 14:17:28.350773 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.350732    2561 scope.go:117] "RemoveContainer" containerID="dcedaa642c66c0caed11041dc781f790b1745958f010bd345c97ca18b185f162"
Jul 17 14:17:28.360487 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.360463    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"]
Jul 17 14:17:28.445411 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.445383    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"]
Jul 17 14:17:28.450340 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.450319    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-988d1f69-openshift-default-56769f64cc-lfzt5"]
Jul 17 14:17:28.773104 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:28.773076    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" path="/var/lib/kubelet/pods/f307dfd6-72f3-48ce-98f0-bcfaaed390ae/volumes"
Jul 17 14:17:29.484285 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:29.484260    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"
Jul 17 14:17:29.628669 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:29.628600    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90b9b65d-2971-44b0-b6ba-a40921ccc64b-maas-api-service-ca\") pod \"90b9b65d-2971-44b0-b6ba-a40921ccc64b\" (UID: \"90b9b65d-2971-44b0-b6ba-a40921ccc64b\") "
Jul 17 14:17:29.629033 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:29.628989    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90b9b65d-2971-44b0-b6ba-a40921ccc64b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "90b9b65d-2971-44b0-b6ba-a40921ccc64b" (UID: "90b9b65d-2971-44b0-b6ba-a40921ccc64b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:17:29.730256 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:29.730222    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90b9b65d-2971-44b0-b6ba-a40921ccc64b-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:17:30.348201 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:30.348167    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl" event={"ID":"90b9b65d-2971-44b0-b6ba-a40921ccc64b","Type":"ContainerDied","Data":"b8f15eb6eeded5d55d0a93f73c4ff26c7a2176321e0a839fe1c809263440620a"}
Jul 17 14:17:30.348201 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:30.348185    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"
Jul 17 14:17:30.348469 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:30.348231    2561 scope.go:117] "RemoveContainer" containerID="4a33a8ac8d37f7c10dc432fdb4766cec1173f89ce1887d1c67c74854ec43e47f"
Jul 17 14:17:30.372083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:30.372050    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"]
Jul 17 14:17:30.377284 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:30.377260    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-64ldl"]
Jul 17 14:17:30.773237 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:30.773206    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" path="/var/lib/kubelet/pods/90b9b65d-2971-44b0-b6ba-a40921ccc64b/volumes"
Jul 17 14:17:35.873510 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.873476    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"]
Jul 17 14:17:35.873929 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.873915    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerName="revoke-keys"
Jul 17 14:17:35.873971 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.873931    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerName="revoke-keys"
Jul 17 14:17:35.873971 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.873951    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy"
Jul 17 14:17:35.873971 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.873957    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy"
Jul 17 14:17:35.873971 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.873962    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerName="revoke-keys"
Jul 17 14:17:35.873971 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.873967    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerName="revoke-keys"
Jul 17 14:17:35.874170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.873993    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy"
Jul 17 14:17:35.874170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.873999    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy"
Jul 17 14:17:35.874170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.874026    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerName="revoke-keys"
Jul 17 14:17:35.874170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.874031    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerName="revoke-keys"
Jul 17 14:17:35.874170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.874099    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy"
Jul 17 14:17:35.874170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.874110    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9715fa58-6a83-4404-b969-94281e33a004" containerName="revoke-keys"
Jul 17 14:17:35.874170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.874116    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerName="revoke-keys"
Jul 17 14:17:35.874170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.874123    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerName="revoke-keys"
Jul 17 14:17:35.874170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.874128    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="90b9b65d-2971-44b0-b6ba-a40921ccc64b" containerName="revoke-keys"
Jul 17 14:17:35.877425 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.877399    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:35.880300 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.880277    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-398a9b19-gw-openshift-default-dockercfg-6gd45\""
Jul 17 14:17:35.893385 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.893361    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"]
Jul 17 14:17:35.991691 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.991656    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-socket\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:35.991866 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.991714    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51797a43-483b-49e4-b2ad-66804cc6e532-istiod-ca-cert\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:35.991866 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.991758    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-data\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:35.991866 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.991796    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-envoy\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:35.991979 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.991862    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jdp2\" (UniqueName: \"kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-kube-api-access-8jdp2\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:35.991979 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.991920    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51797a43-483b-49e4-b2ad-66804cc6e532-istio-podinfo\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:35.992083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.991983    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-istio-token\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:35.992083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.992070    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-credential-socket\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:35.992158 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:35.992138    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-certs\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093106 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093075    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-certs\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093289 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093141    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-socket\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093289 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093173    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51797a43-483b-49e4-b2ad-66804cc6e532-istiod-ca-cert\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093289 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093204    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-data\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093289 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093234    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-envoy\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093289 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093262    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jdp2\" (UniqueName: \"kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-kube-api-access-8jdp2\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093562 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51797a43-483b-49e4-b2ad-66804cc6e532-istio-podinfo\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093562 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093354    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-istio-token\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093562 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093393    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-credential-socket\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093562 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093530    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-certs\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093562 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093557    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-socket\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093821 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093635    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-data\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093821 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093761    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-credential-socket\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.093911 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.093892    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51797a43-483b-49e4-b2ad-66804cc6e532-istiod-ca-cert\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.095690 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.095668    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-envoy\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.096175 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.096157    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51797a43-483b-49e4-b2ad-66804cc6e532-istio-podinfo\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.103233 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.103211    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-istio-token\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.103410 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.103388    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jdp2\" (UniqueName: \"kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-kube-api-access-8jdp2\") pod \"e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") " pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.188155 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.188138    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:36.338575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.338552    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"]
Jul 17 14:17:36.340386 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:17:36.340358    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51797a43_483b_49e4_b2ad_66804cc6e532.slice/crio-21b7434ad806bff63c17bc887a797b174c801fe1a34dc3046a3d3d18343f557a WatchSource:0}: Error finding container 21b7434ad806bff63c17bc887a797b174c801fe1a34dc3046a3d3d18343f557a: Status 404 returned error can't find the container with id 21b7434ad806bff63c17bc887a797b174c801fe1a34dc3046a3d3d18343f557a
Jul 17 14:17:36.342719 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.342680    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:17:36.342801 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.342779    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:17:36.342841 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.342822    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:17:36.387581 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:36.387554    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" event={"ID":"51797a43-483b-49e4-b2ad-66804cc6e532","Type":"ContainerStarted","Data":"21b7434ad806bff63c17bc887a797b174c801fe1a34dc3046a3d3d18343f557a"}
Jul 17 14:17:37.394502 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:37.394468    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" event={"ID":"51797a43-483b-49e4-b2ad-66804cc6e532","Type":"ContainerStarted","Data":"fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c"}
Jul 17 14:17:37.416966 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:37.416919    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" podStartSLOduration=2.4169015480000002 podStartE2EDuration="2.416901548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:17:37.41515424 +0000 UTC m=+1943.121601075" watchObservedRunningTime="2026-07-17 14:17:37.416901548 +0000 UTC m=+1943.123348382"
Jul 17 14:17:38.188626 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:38.188591    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:38.194025 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:38.193991    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:38.400378 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:38.400352    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:38.401228 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:38.401212    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:41.662037 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:41.661991    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"]
Jul 17 14:17:41.662514 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:41.662498    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f307dfd6-72f3-48ce-98f0-bcfaaed390ae" containerName="istio-proxy"
Jul 17 14:17:41.665844 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:41.665825    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"
Jul 17 14:17:41.669061 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:41.669040    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:17:41.669149 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:41.669119    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:17:41.675028 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:41.674978    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"]
Jul 17 14:17:41.756486 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:41.756457    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5600a09a-4275-4c55-a401-6dda98adb30f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt\" (UID: \"5600a09a-4275-4c55-a401-6dda98adb30f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"
Jul 17 14:17:41.857545 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:41.857509    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5600a09a-4275-4c55-a401-6dda98adb30f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt\" (UID: \"5600a09a-4275-4c55-a401-6dda98adb30f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"
Jul 17 14:17:41.858142 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:41.858123    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5600a09a-4275-4c55-a401-6dda98adb30f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt\" (UID: \"5600a09a-4275-4c55-a401-6dda98adb30f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"
Jul 17 14:17:41.977774 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:41.977742    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"
Jul 17 14:17:42.155586 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:42.155550    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"]
Jul 17 14:17:42.158077 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:17:42.158044    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5600a09a_4275_4c55_a401_6dda98adb30f.slice/crio-2c41673fa986a41f95b95996b8066b2db34ea06cfdc0a5435b96345841cc2545 WatchSource:0}: Error finding container 2c41673fa986a41f95b95996b8066b2db34ea06cfdc0a5435b96345841cc2545: Status 404 returned error can't find the container with id 2c41673fa986a41f95b95996b8066b2db34ea06cfdc0a5435b96345841cc2545
Jul 17 14:17:42.421413 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:42.421369    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt" event={"ID":"5600a09a-4275-4c55-a401-6dda98adb30f","Type":"ContainerStarted","Data":"2c41673fa986a41f95b95996b8066b2db34ea06cfdc0a5435b96345841cc2545"}
Jul 17 14:17:43.431778 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:43.431735    2561 generic.go:356] "Generic (PLEG): container finished" podID="5600a09a-4275-4c55-a401-6dda98adb30f" containerID="04f94a53a45cc80858482586d7c189e9c951078456b8435cd64e65cb4b0945ee" exitCode=7
Jul 17 14:17:43.432296 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:43.431858    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt" event={"ID":"5600a09a-4275-4c55-a401-6dda98adb30f","Type":"ContainerDied","Data":"04f94a53a45cc80858482586d7c189e9c951078456b8435cd64e65cb4b0945ee"}
Jul 17 14:17:43.432296 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:43.432216    2561 scope.go:117] "RemoveContainer" containerID="04f94a53a45cc80858482586d7c189e9c951078456b8435cd64e65cb4b0945ee"
Jul 17 14:17:44.443181 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:44.443141    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt" event={"ID":"5600a09a-4275-4c55-a401-6dda98adb30f","Type":"ContainerStarted","Data":"f0dc7166e2c746f8847674680b0a4dc582825d48dd90b0736ec71756a1fed6d3"}
Jul 17 14:17:44.443612 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:44.443587    2561 scope.go:117] "RemoveContainer" containerID="f0dc7166e2c746f8847674680b0a4dc582825d48dd90b0736ec71756a1fed6d3"
Jul 17 14:17:44.443876 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:17:44.443848    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt_odh-ai-gateway-infra(5600a09a-4275-4c55-a401-6dda98adb30f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt" podUID="5600a09a-4275-4c55-a401-6dda98adb30f"
Jul 17 14:17:45.451876 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:45.451840    2561 generic.go:356] "Generic (PLEG): container finished" podID="5600a09a-4275-4c55-a401-6dda98adb30f" containerID="f0dc7166e2c746f8847674680b0a4dc582825d48dd90b0736ec71756a1fed6d3" exitCode=7
Jul 17 14:17:45.452333 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:45.451947    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt" event={"ID":"5600a09a-4275-4c55-a401-6dda98adb30f","Type":"ContainerDied","Data":"f0dc7166e2c746f8847674680b0a4dc582825d48dd90b0736ec71756a1fed6d3"}
Jul 17 14:17:45.452333 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:45.451994    2561 scope.go:117] "RemoveContainer" containerID="04f94a53a45cc80858482586d7c189e9c951078456b8435cd64e65cb4b0945ee"
Jul 17 14:17:45.452333 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:45.452257    2561 scope.go:117] "RemoveContainer" containerID="f0dc7166e2c746f8847674680b0a4dc582825d48dd90b0736ec71756a1fed6d3"
Jul 17 14:17:45.452603 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:17:45.452565    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt_odh-ai-gateway-infra(5600a09a-4275-4c55-a401-6dda98adb30f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt" podUID="5600a09a-4275-4c55-a401-6dda98adb30f"
Jul 17 14:17:46.459511 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:46.459483    2561 scope.go:117] "RemoveContainer" containerID="f0dc7166e2c746f8847674680b0a4dc582825d48dd90b0736ec71756a1fed6d3"
Jul 17 14:17:46.460043 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:17:46.459705    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt_odh-ai-gateway-infra(5600a09a-4275-4c55-a401-6dda98adb30f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt" podUID="5600a09a-4275-4c55-a401-6dda98adb30f"
Jul 17 14:17:48.472100 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:48.472068    2561 generic.go:356] "Generic (PLEG): container finished" podID="51797a43-483b-49e4-b2ad-66804cc6e532" containerID="fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c" exitCode=0
Jul 17 14:17:48.472517 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:48.472153    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" event={"ID":"51797a43-483b-49e4-b2ad-66804cc6e532","Type":"ContainerDied","Data":"fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c"}
Jul 17 14:17:48.472567 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:48.472534    2561 scope.go:117] "RemoveContainer" containerID="fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c"
Jul 17 14:17:48.473553 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:48.473527    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:17:48.473617 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:48.473603    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:17:48.473657 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:48.473636    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:17:49.479690 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:49.479655    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" event={"ID":"51797a43-483b-49e4-b2ad-66804cc6e532","Type":"ContainerStarted","Data":"8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68"}
Jul 17 14:17:50.189277 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:50.189224    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:50.190519 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:50.190490    2561 patch_prober.go:28] interesting pod/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 17 14:17:50.190674 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:50.190551    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 17 14:17:51.188675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:51.188634    2561 patch_prober.go:28] interesting pod/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 17 14:17:51.189147 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:51.188705    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 17 14:17:52.189328 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:52.189293    2561 patch_prober.go:28] interesting pod/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 17 14:17:52.189732 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:52.189349    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 17 14:17:53.188745 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:53.188704    2561 patch_prober.go:28] interesting pod/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 17 14:17:53.188926 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:53.188768    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 17 14:17:53.400993 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:53.400962    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:54.192797 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:54.192769    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:54.193912 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:54.193889    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:17:57.766036 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:57.765981    2561 scope.go:117] "RemoveContainer" containerID="f0dc7166e2c746f8847674680b0a4dc582825d48dd90b0736ec71756a1fed6d3"
Jul 17 14:17:58.537952 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:58.537914    2561 generic.go:356] "Generic (PLEG): container finished" podID="5600a09a-4275-4c55-a401-6dda98adb30f" containerID="3c208fc8fd83574a11fd5b2975f0ae3f1d71ad7a839804de55ba564b06cc791a" exitCode=7
Jul 17 14:17:58.537952 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:58.537933    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt" event={"ID":"5600a09a-4275-4c55-a401-6dda98adb30f","Type":"ContainerDied","Data":"3c208fc8fd83574a11fd5b2975f0ae3f1d71ad7a839804de55ba564b06cc791a"}
Jul 17 14:17:58.538232 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:58.538018    2561 scope.go:117] "RemoveContainer" containerID="f0dc7166e2c746f8847674680b0a4dc582825d48dd90b0736ec71756a1fed6d3"
Jul 17 14:17:58.538305 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:58.538290    2561 scope.go:117] "RemoveContainer" containerID="3c208fc8fd83574a11fd5b2975f0ae3f1d71ad7a839804de55ba564b06cc791a"
Jul 17 14:17:58.538518 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:17:58.538498    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt_odh-ai-gateway-infra(5600a09a-4275-4c55-a401-6dda98adb30f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt" podUID="5600a09a-4275-4c55-a401-6dda98adb30f"
Jul 17 14:17:59.565602 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:59.565570    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"]
Jul 17 14:17:59.713295 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:59.713262    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"
Jul 17 14:17:59.833723 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:59.833648    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5600a09a-4275-4c55-a401-6dda98adb30f-maas-api-service-ca\") pod \"5600a09a-4275-4c55-a401-6dda98adb30f\" (UID: \"5600a09a-4275-4c55-a401-6dda98adb30f\") "
Jul 17 14:17:59.834121 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:59.834083    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5600a09a-4275-4c55-a401-6dda98adb30f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5600a09a-4275-4c55-a401-6dda98adb30f" (UID: "5600a09a-4275-4c55-a401-6dda98adb30f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:17:59.935078 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:17:59.935046    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5600a09a-4275-4c55-a401-6dda98adb30f-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:00.551691 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:00.551663    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt" event={"ID":"5600a09a-4275-4c55-a401-6dda98adb30f","Type":"ContainerDied","Data":"2c41673fa986a41f95b95996b8066b2db34ea06cfdc0a5435b96345841cc2545"}
Jul 17 14:18:00.551892 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:00.551717    2561 scope.go:117] "RemoveContainer" containerID="3c208fc8fd83574a11fd5b2975f0ae3f1d71ad7a839804de55ba564b06cc791a"
Jul 17 14:18:00.551892 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:00.551669    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"
Jul 17 14:18:00.572544 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:18:00.572517    2561 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5600a09a_4275_4c55_a401_6dda98adb30f.slice\": RecentStats: unable to find data in memory cache]"
Jul 17 14:18:00.579052 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:00.579029    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"]
Jul 17 14:18:00.585276 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:00.585255    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-gp7pt"]
Jul 17 14:18:00.771907 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:00.771877    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5600a09a-4275-4c55-a401-6dda98adb30f" path="/var/lib/kubelet/pods/5600a09a-4275-4c55-a401-6dda98adb30f/volumes"
Jul 17 14:18:02.610918 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.610885    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"]
Jul 17 14:18:02.611545 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.611525    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5600a09a-4275-4c55-a401-6dda98adb30f" containerName="revoke-keys"
Jul 17 14:18:02.611643 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.611549    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5600a09a-4275-4c55-a401-6dda98adb30f" containerName="revoke-keys"
Jul 17 14:18:02.611643 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.611596    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5600a09a-4275-4c55-a401-6dda98adb30f" containerName="revoke-keys"
Jul 17 14:18:02.611643 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.611607    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5600a09a-4275-4c55-a401-6dda98adb30f" containerName="revoke-keys"
Jul 17 14:18:02.611792 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.611708    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5600a09a-4275-4c55-a401-6dda98adb30f" containerName="revoke-keys"
Jul 17 14:18:02.611792 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.611719    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5600a09a-4275-4c55-a401-6dda98adb30f" containerName="revoke-keys"
Jul 17 14:18:02.611792 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.611731    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5600a09a-4275-4c55-a401-6dda98adb30f" containerName="revoke-keys"
Jul 17 14:18:02.614562 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.614544    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"
Jul 17 14:18:02.617281 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.617262    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:18:02.617281 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.617273    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:18:02.622907 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.622886    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"]
Jul 17 14:18:02.661700 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.661674    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df7778d3-6657-47fd-be94-30e21b58ed75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw\" (UID: \"df7778d3-6657-47fd-be94-30e21b58ed75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"
Jul 17 14:18:02.763175 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.763148    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df7778d3-6657-47fd-be94-30e21b58ed75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw\" (UID: \"df7778d3-6657-47fd-be94-30e21b58ed75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"
Jul 17 14:18:02.763757 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.763736    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df7778d3-6657-47fd-be94-30e21b58ed75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw\" (UID: \"df7778d3-6657-47fd-be94-30e21b58ed75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"
Jul 17 14:18:02.927289 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:02.927266    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"
Jul 17 14:18:03.070924 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:03.070899    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"]
Jul 17 14:18:03.072738 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:18:03.072709    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf7778d3_6657_47fd_be94_30e21b58ed75.slice/crio-6e74b6025929c6dcb286e1bbcb70f0ef8daa88a3343d134001c0f81d4fbdc1cb WatchSource:0}: Error finding container 6e74b6025929c6dcb286e1bbcb70f0ef8daa88a3343d134001c0f81d4fbdc1cb: Status 404 returned error can't find the container with id 6e74b6025929c6dcb286e1bbcb70f0ef8daa88a3343d134001c0f81d4fbdc1cb
Jul 17 14:18:03.570382 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:03.570302    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw" event={"ID":"df7778d3-6657-47fd-be94-30e21b58ed75","Type":"ContainerStarted","Data":"879dc7363a4432657b26d145db15ca15522b4401aed41fa9eed19a3dcd62e56a"}
Jul 17 14:18:03.570382 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:03.570381    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw" event={"ID":"df7778d3-6657-47fd-be94-30e21b58ed75","Type":"ContainerStarted","Data":"6e74b6025929c6dcb286e1bbcb70f0ef8daa88a3343d134001c0f81d4fbdc1cb"}
Jul 17 14:18:03.590609 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:03.590565    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw" podStartSLOduration=1.590551437 podStartE2EDuration="1.590551437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:18:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:18:03.588364328 +0000 UTC m=+1969.294811175" watchObservedRunningTime="2026-07-17 14:18:03.590551437 +0000 UTC m=+1969.296998271"
Jul 17 14:18:04.577499 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:04.577463    2561 generic.go:356] "Generic (PLEG): container finished" podID="df7778d3-6657-47fd-be94-30e21b58ed75" containerID="879dc7363a4432657b26d145db15ca15522b4401aed41fa9eed19a3dcd62e56a" exitCode=6
Jul 17 14:18:04.577863 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:04.577558    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw" event={"ID":"df7778d3-6657-47fd-be94-30e21b58ed75","Type":"ContainerDied","Data":"879dc7363a4432657b26d145db15ca15522b4401aed41fa9eed19a3dcd62e56a"}
Jul 17 14:18:04.577863 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:04.577752    2561 scope.go:117] "RemoveContainer" containerID="879dc7363a4432657b26d145db15ca15522b4401aed41fa9eed19a3dcd62e56a"
Jul 17 14:18:05.584886 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:05.584812    2561 generic.go:356] "Generic (PLEG): container finished" podID="df7778d3-6657-47fd-be94-30e21b58ed75" containerID="3d1d95e0d15e63ef22f9f97e78dfcaf13b4efd819d6b296e127270c43f3250c3" exitCode=6
Jul 17 14:18:05.585340 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:05.584903    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw" event={"ID":"df7778d3-6657-47fd-be94-30e21b58ed75","Type":"ContainerDied","Data":"3d1d95e0d15e63ef22f9f97e78dfcaf13b4efd819d6b296e127270c43f3250c3"}
Jul 17 14:18:05.585340 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:05.584970    2561 scope.go:117] "RemoveContainer" containerID="879dc7363a4432657b26d145db15ca15522b4401aed41fa9eed19a3dcd62e56a"
Jul 17 14:18:05.585340 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:05.585104    2561 scope.go:117] "RemoveContainer" containerID="3d1d95e0d15e63ef22f9f97e78dfcaf13b4efd819d6b296e127270c43f3250c3"
Jul 17 14:18:05.585340 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:18:05.585310    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw_odh-ai-gateway-infra(df7778d3-6657-47fd-be94-30e21b58ed75)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw" podUID="df7778d3-6657-47fd-be94-30e21b58ed75"
Jul 17 14:18:06.592284 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:06.592252    2561 scope.go:117] "RemoveContainer" containerID="3d1d95e0d15e63ef22f9f97e78dfcaf13b4efd819d6b296e127270c43f3250c3"
Jul 17 14:18:06.592690 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:18:06.592463    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw_odh-ai-gateway-infra(df7778d3-6657-47fd-be94-30e21b58ed75)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw" podUID="df7778d3-6657-47fd-be94-30e21b58ed75"
Jul 17 14:18:17.765794 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:17.765766    2561 scope.go:117] "RemoveContainer" containerID="3d1d95e0d15e63ef22f9f97e78dfcaf13b4efd819d6b296e127270c43f3250c3"
Jul 17 14:18:18.656820 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:18.656787    2561 generic.go:356] "Generic (PLEG): container finished" podID="df7778d3-6657-47fd-be94-30e21b58ed75" containerID="115c580ffb64cae88884b3f33eefc30f7eefc7aa76504011547dd63e7f6999d4" exitCode=6
Jul 17 14:18:18.656979 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:18.656831    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw" event={"ID":"df7778d3-6657-47fd-be94-30e21b58ed75","Type":"ContainerDied","Data":"115c580ffb64cae88884b3f33eefc30f7eefc7aa76504011547dd63e7f6999d4"}
Jul 17 14:18:18.656979 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:18.656887    2561 scope.go:117] "RemoveContainer" containerID="3d1d95e0d15e63ef22f9f97e78dfcaf13b4efd819d6b296e127270c43f3250c3"
Jul 17 14:18:18.657199 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:18.657182    2561 scope.go:117] "RemoveContainer" containerID="115c580ffb64cae88884b3f33eefc30f7eefc7aa76504011547dd63e7f6999d4"
Jul 17 14:18:18.657439 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:18:18.657418    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw_odh-ai-gateway-infra(df7778d3-6657-47fd-be94-30e21b58ed75)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw" podUID="df7778d3-6657-47fd-be94-30e21b58ed75"
Jul 17 14:18:19.684854 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:19.684822    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"]
Jul 17 14:18:19.822984 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:19.822962    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"
Jul 17 14:18:19.917569 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:19.917540    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df7778d3-6657-47fd-be94-30e21b58ed75-maas-api-service-ca\") pod \"df7778d3-6657-47fd-be94-30e21b58ed75\" (UID: \"df7778d3-6657-47fd-be94-30e21b58ed75\") "
Jul 17 14:18:19.917883 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:19.917858    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df7778d3-6657-47fd-be94-30e21b58ed75-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "df7778d3-6657-47fd-be94-30e21b58ed75" (UID: "df7778d3-6657-47fd-be94-30e21b58ed75"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:18:20.018877 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:20.018811    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df7778d3-6657-47fd-be94-30e21b58ed75-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:20.669268 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:20.669236    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"
Jul 17 14:18:20.669268 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:20.669244    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw" event={"ID":"df7778d3-6657-47fd-be94-30e21b58ed75","Type":"ContainerDied","Data":"6e74b6025929c6dcb286e1bbcb70f0ef8daa88a3343d134001c0f81d4fbdc1cb"}
Jul 17 14:18:20.669501 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:20.669297    2561 scope.go:117] "RemoveContainer" containerID="115c580ffb64cae88884b3f33eefc30f7eefc7aa76504011547dd63e7f6999d4"
Jul 17 14:18:20.697377 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:20.697350    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"]
Jul 17 14:18:20.713301 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:20.713272    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-w7wqw"]
Jul 17 14:18:20.772063 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:20.772037    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df7778d3-6657-47fd-be94-30e21b58ed75" path="/var/lib/kubelet/pods/df7778d3-6657-47fd-be94-30e21b58ed75/volumes"
Jul 17 14:18:31.704808 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.704777    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"]
Jul 17 14:18:31.705239 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.705226    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5600a09a-4275-4c55-a401-6dda98adb30f" containerName="revoke-keys"
Jul 17 14:18:31.705239 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.705240    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5600a09a-4275-4c55-a401-6dda98adb30f" containerName="revoke-keys"
Jul 17 14:18:31.705314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.705247    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df7778d3-6657-47fd-be94-30e21b58ed75" containerName="revoke-keys"
Jul 17 14:18:31.705314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.705252    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="df7778d3-6657-47fd-be94-30e21b58ed75" containerName="revoke-keys"
Jul 17 14:18:31.705314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.705266    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df7778d3-6657-47fd-be94-30e21b58ed75" containerName="revoke-keys"
Jul 17 14:18:31.705314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.705273    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="df7778d3-6657-47fd-be94-30e21b58ed75" containerName="revoke-keys"
Jul 17 14:18:31.705314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.705282    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df7778d3-6657-47fd-be94-30e21b58ed75" containerName="revoke-keys"
Jul 17 14:18:31.705314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.705288    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="df7778d3-6657-47fd-be94-30e21b58ed75" containerName="revoke-keys"
Jul 17 14:18:31.705491 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.705347    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="df7778d3-6657-47fd-be94-30e21b58ed75" containerName="revoke-keys"
Jul 17 14:18:31.705491 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.705355    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="df7778d3-6657-47fd-be94-30e21b58ed75" containerName="revoke-keys"
Jul 17 14:18:31.708354 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.708338    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"
Jul 17 14:18:31.711070 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.711048    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:18:31.711234 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.711213    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:18:31.716192 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.716168    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"]
Jul 17 14:18:31.822753 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.822723    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a69359ed-0fb8-44a2-95ef-561994dab94b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c\" (UID: \"a69359ed-0fb8-44a2-95ef-561994dab94b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"
Jul 17 14:18:31.923392 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.923366    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a69359ed-0fb8-44a2-95ef-561994dab94b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c\" (UID: \"a69359ed-0fb8-44a2-95ef-561994dab94b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"
Jul 17 14:18:31.923927 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:31.923909    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a69359ed-0fb8-44a2-95ef-561994dab94b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c\" (UID: \"a69359ed-0fb8-44a2-95ef-561994dab94b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"
Jul 17 14:18:32.020889 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:32.020820    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"
Jul 17 14:18:32.154439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:32.154415    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"]
Jul 17 14:18:32.155938 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:18:32.155912    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69359ed_0fb8_44a2_95ef_561994dab94b.slice/crio-bc119c3ec5f7cdc10c0aee055d5172b821196f27b0bd4671155f1c66272f5a0f WatchSource:0}: Error finding container bc119c3ec5f7cdc10c0aee055d5172b821196f27b0bd4671155f1c66272f5a0f: Status 404 returned error can't find the container with id bc119c3ec5f7cdc10c0aee055d5172b821196f27b0bd4671155f1c66272f5a0f
Jul 17 14:18:32.736246 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:32.736220    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c" event={"ID":"a69359ed-0fb8-44a2-95ef-561994dab94b","Type":"ContainerStarted","Data":"5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee"}
Jul 17 14:18:32.736584 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:32.736281    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c" event={"ID":"a69359ed-0fb8-44a2-95ef-561994dab94b","Type":"ContainerStarted","Data":"bc119c3ec5f7cdc10c0aee055d5172b821196f27b0bd4671155f1c66272f5a0f"}
Jul 17 14:18:32.736584 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:32.736476    2561 scope.go:117] "RemoveContainer" containerID="5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee"
Jul 17 14:18:33.743950 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:33.743912    2561 generic.go:356] "Generic (PLEG): container finished" podID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerID="5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee" exitCode=7
Jul 17 14:18:33.743950 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:33.743943    2561 generic.go:356] "Generic (PLEG): container finished" podID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerID="94462dc0a199bcb8f51390a583c09afb5013a0ad9a1630f56ab442522e0694d0" exitCode=7
Jul 17 14:18:33.744401 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:33.743996    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c" event={"ID":"a69359ed-0fb8-44a2-95ef-561994dab94b","Type":"ContainerDied","Data":"5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee"}
Jul 17 14:18:33.744401 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:33.744096    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c" event={"ID":"a69359ed-0fb8-44a2-95ef-561994dab94b","Type":"ContainerDied","Data":"94462dc0a199bcb8f51390a583c09afb5013a0ad9a1630f56ab442522e0694d0"}
Jul 17 14:18:33.744401 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:33.744130    2561 scope.go:117] "RemoveContainer" containerID="5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee"
Jul 17 14:18:33.744401 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:33.744306    2561 scope.go:117] "RemoveContainer" containerID="94462dc0a199bcb8f51390a583c09afb5013a0ad9a1630f56ab442522e0694d0"
Jul 17 14:18:33.744584 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:18:33.744537    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c_odh-ai-gateway-infra(a69359ed-0fb8-44a2-95ef-561994dab94b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b"
Jul 17 14:18:33.755136 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:33.755109    2561 scope.go:117] "RemoveContainer" containerID="5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee"
Jul 17 14:18:33.755388 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:18:33.755369    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee\": container with ID starting with 5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee not found: ID does not exist" containerID="5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee"
Jul 17 14:18:33.755436 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:33.755397    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee"} err="failed to get container status \"5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee\": rpc error: code = NotFound desc = could not find container \"5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee\": container with ID starting with 5371d0d0f1877bc3db036aff07f4bdbf71f4c40e2f08a7c468af8d0f5913f1ee not found: ID does not exist"
Jul 17 14:18:34.752216 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:34.752183    2561 scope.go:117] "RemoveContainer" containerID="94462dc0a199bcb8f51390a583c09afb5013a0ad9a1630f56ab442522e0694d0"
Jul 17 14:18:34.752602 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:18:34.752355    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c_odh-ai-gateway-infra(a69359ed-0fb8-44a2-95ef-561994dab94b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b"
Jul 17 14:18:41.783172 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:41.783134    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"]
Jul 17 14:18:41.783588 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:41.783325    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy" containerID="cri-o://8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68" gracePeriod=30
Jul 17 14:18:45.765941 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:45.765913    2561 scope.go:117] "RemoveContainer" containerID="94462dc0a199bcb8f51390a583c09afb5013a0ad9a1630f56ab442522e0694d0"
Jul 17 14:18:46.816083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:46.816056    2561 generic.go:356] "Generic (PLEG): container finished" podID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerID="d2b7d4493b6ef45b03b83f584fb25a31235b048e14561404f4bfd531729dfeef" exitCode=7
Jul 17 14:18:46.816480 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:46.816144    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c" event={"ID":"a69359ed-0fb8-44a2-95ef-561994dab94b","Type":"ContainerDied","Data":"d2b7d4493b6ef45b03b83f584fb25a31235b048e14561404f4bfd531729dfeef"}
Jul 17 14:18:46.816480 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:46.816225    2561 scope.go:117] "RemoveContainer" containerID="94462dc0a199bcb8f51390a583c09afb5013a0ad9a1630f56ab442522e0694d0"
Jul 17 14:18:46.816573 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:46.816541    2561 scope.go:117] "RemoveContainer" containerID="d2b7d4493b6ef45b03b83f584fb25a31235b048e14561404f4bfd531729dfeef"
Jul 17 14:18:46.816814 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:18:46.816793    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c_odh-ai-gateway-infra(a69359ed-0fb8-44a2-95ef-561994dab94b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b"
Jul 17 14:18:47.049674 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.049612    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:18:47.162686 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.162654    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-credential-socket\") pod \"51797a43-483b-49e4-b2ad-66804cc6e532\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") "
Jul 17 14:18:47.162860 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.162708    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jdp2\" (UniqueName: \"kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-kube-api-access-8jdp2\") pod \"51797a43-483b-49e4-b2ad-66804cc6e532\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") "
Jul 17 14:18:47.162860 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.162745    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51797a43-483b-49e4-b2ad-66804cc6e532-istiod-ca-cert\") pod \"51797a43-483b-49e4-b2ad-66804cc6e532\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") "
Jul 17 14:18:47.162860 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.162774    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-istio-token\") pod \"51797a43-483b-49e4-b2ad-66804cc6e532\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") "
Jul 17 14:18:47.162860 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.162803    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51797a43-483b-49e4-b2ad-66804cc6e532-istio-podinfo\") pod \"51797a43-483b-49e4-b2ad-66804cc6e532\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") "
Jul 17 14:18:47.162860 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.162835    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-data\") pod \"51797a43-483b-49e4-b2ad-66804cc6e532\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") "
Jul 17 14:18:47.163168 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.162915    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "51797a43-483b-49e4-b2ad-66804cc6e532" (UID: "51797a43-483b-49e4-b2ad-66804cc6e532"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:18:47.163532 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.163436    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-certs\") pod \"51797a43-483b-49e4-b2ad-66804cc6e532\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") "
Jul 17 14:18:47.163532 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.163523    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-socket\") pod \"51797a43-483b-49e4-b2ad-66804cc6e532\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") "
Jul 17 14:18:47.163872 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.163611    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-envoy\") pod \"51797a43-483b-49e4-b2ad-66804cc6e532\" (UID: \"51797a43-483b-49e4-b2ad-66804cc6e532\") "
Jul 17 14:18:47.163872 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.163777    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-data" (OuterVolumeSpecName: "istio-data") pod "51797a43-483b-49e4-b2ad-66804cc6e532" (UID: "51797a43-483b-49e4-b2ad-66804cc6e532"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:18:47.164055 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.164032    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "51797a43-483b-49e4-b2ad-66804cc6e532" (UID: "51797a43-483b-49e4-b2ad-66804cc6e532"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:18:47.164431 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.164142    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:47.164431 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.164170    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:47.164431 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.164185    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:47.164431 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.164199    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "51797a43-483b-49e4-b2ad-66804cc6e532" (UID: "51797a43-483b-49e4-b2ad-66804cc6e532"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:18:47.164431 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.164367    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51797a43-483b-49e4-b2ad-66804cc6e532-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "51797a43-483b-49e4-b2ad-66804cc6e532" (UID: "51797a43-483b-49e4-b2ad-66804cc6e532"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:18:47.166446 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.166422    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-kube-api-access-8jdp2" (OuterVolumeSpecName: "kube-api-access-8jdp2") pod "51797a43-483b-49e4-b2ad-66804cc6e532" (UID: "51797a43-483b-49e4-b2ad-66804cc6e532"). InnerVolumeSpecName "kube-api-access-8jdp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:18:47.166708 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.166685    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-istio-token" (OuterVolumeSpecName: "istio-token") pod "51797a43-483b-49e4-b2ad-66804cc6e532" (UID: "51797a43-483b-49e4-b2ad-66804cc6e532"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:18:47.167143 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.167118    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51797a43-483b-49e4-b2ad-66804cc6e532-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "51797a43-483b-49e4-b2ad-66804cc6e532" (UID: "51797a43-483b-49e4-b2ad-66804cc6e532"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:18:47.168110 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.168080    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "51797a43-483b-49e4-b2ad-66804cc6e532" (UID: "51797a43-483b-49e4-b2ad-66804cc6e532"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:18:47.265517 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.265491    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:47.265517 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.265517    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jdp2\" (UniqueName: \"kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-kube-api-access-8jdp2\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:47.265702 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.265531    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51797a43-483b-49e4-b2ad-66804cc6e532-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:47.265702 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.265543    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51797a43-483b-49e4-b2ad-66804cc6e532-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:47.265702 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.265554    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51797a43-483b-49e4-b2ad-66804cc6e532-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:47.265702 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.265564    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51797a43-483b-49e4-b2ad-66804cc6e532-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:47.823385 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.823352    2561 generic.go:356] "Generic (PLEG): container finished" podID="51797a43-483b-49e4-b2ad-66804cc6e532" containerID="8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68" exitCode=0
Jul 17 14:18:47.823828 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.823418    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"
Jul 17 14:18:47.823828 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.823442    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" event={"ID":"51797a43-483b-49e4-b2ad-66804cc6e532","Type":"ContainerDied","Data":"8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68"}
Jul 17 14:18:47.823828 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.823499    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt" event={"ID":"51797a43-483b-49e4-b2ad-66804cc6e532","Type":"ContainerDied","Data":"21b7434ad806bff63c17bc887a797b174c801fe1a34dc3046a3d3d18343f557a"}
Jul 17 14:18:47.823828 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.823531    2561 scope.go:117] "RemoveContainer" containerID="8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68"
Jul 17 14:18:47.833424 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.833406    2561 scope.go:117] "RemoveContainer" containerID="fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c"
Jul 17 14:18:47.842403 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.842385    2561 scope.go:117] "RemoveContainer" containerID="8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68"
Jul 17 14:18:47.842659 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:18:47.842635    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68\": container with ID starting with 8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68 not found: ID does not exist" containerID="8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68"
Jul 17 14:18:47.842724 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.842666    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68"} err="failed to get container status \"8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68\": rpc error: code = NotFound desc = could not find container \"8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68\": container with ID starting with 8b8361362e86e6cb54d93be2f96bb34a449458c4d3856d4921b6887778d1bb68 not found: ID does not exist"
Jul 17 14:18:47.842724 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.842684    2561 scope.go:117] "RemoveContainer" containerID="fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c"
Jul 17 14:18:47.842939 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:18:47.842918    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c\": container with ID starting with fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c not found: ID does not exist" containerID="fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c"
Jul 17 14:18:47.843049 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.842946    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c"} err="failed to get container status \"fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c\": rpc error: code = NotFound desc = could not find container \"fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c\": container with ID starting with fa0c7cd855148155a04fe4d72286769752955504cd29a5a54c07ac8031eaae6c not found: ID does not exist"
Jul 17 14:18:47.843970 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.843948    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"]
Jul 17 14:18:47.850376 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.850355    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"]
Jul 17 14:18:47.856190 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.856164    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-398a9b19-gw-openshift-default-6fd797d98c-ljxnt"]
Jul 17 14:18:47.984718 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:47.984696    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"
Jul 17 14:18:48.073591 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:48.073522    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a69359ed-0fb8-44a2-95ef-561994dab94b-maas-api-service-ca\") pod \"a69359ed-0fb8-44a2-95ef-561994dab94b\" (UID: \"a69359ed-0fb8-44a2-95ef-561994dab94b\") "
Jul 17 14:18:48.073850 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:48.073827    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a69359ed-0fb8-44a2-95ef-561994dab94b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a69359ed-0fb8-44a2-95ef-561994dab94b" (UID: "a69359ed-0fb8-44a2-95ef-561994dab94b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:18:48.174597 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:48.174571    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a69359ed-0fb8-44a2-95ef-561994dab94b-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:18:48.773176 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:48.773146    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" path="/var/lib/kubelet/pods/51797a43-483b-49e4-b2ad-66804cc6e532/volumes"
Jul 17 14:18:48.833880 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:48.833848    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c" event={"ID":"a69359ed-0fb8-44a2-95ef-561994dab94b","Type":"ContainerDied","Data":"bc119c3ec5f7cdc10c0aee055d5172b821196f27b0bd4671155f1c66272f5a0f"}
Jul 17 14:18:48.833880 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:48.833881    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"
Jul 17 14:18:48.834343 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:48.833914    2561 scope.go:117] "RemoveContainer" containerID="d2b7d4493b6ef45b03b83f584fb25a31235b048e14561404f4bfd531729dfeef"
Jul 17 14:18:48.853703 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:48.853674    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"]
Jul 17 14:18:48.858411 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:48.858389    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-dc46c"]
Jul 17 14:18:50.771832 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:50.771791    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b" path="/var/lib/kubelet/pods/a69359ed-0fb8-44a2-95ef-561994dab94b/volumes"
Jul 17 14:18:54.497832 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.497801    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"]
Jul 17 14:18:54.498283 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498268    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy"
Jul 17 14:18:54.498283 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498284    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy"
Jul 17 14:18:54.498354 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498308    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerName="revoke-keys"
Jul 17 14:18:54.498354 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498314    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerName="revoke-keys"
Jul 17 14:18:54.498354 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498324    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerName="revoke-keys"
Jul 17 14:18:54.498354 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498329    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerName="revoke-keys"
Jul 17 14:18:54.498354 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498336    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerName="revoke-keys"
Jul 17 14:18:54.498354 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498342    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerName="revoke-keys"
Jul 17 14:18:54.498552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498411    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerName="revoke-keys"
Jul 17 14:18:54.498552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498418    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy"
Jul 17 14:18:54.498552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498427    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy"
Jul 17 14:18:54.498552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498434    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="df7778d3-6657-47fd-be94-30e21b58ed75" containerName="revoke-keys"
Jul 17 14:18:54.498552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.498440    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerName="revoke-keys"
Jul 17 14:18:54.503069 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.503048    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.506029 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.505991    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d7874222-openshift-default-dockercfg-bnm2l\""
Jul 17 14:18:54.516632 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.516607    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"]
Jul 17 14:18:54.633707 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.633670    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-token\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.633707 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.633707    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-certs\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.633927 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.633798    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-podinfo\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.633927 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.633833    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-credential-socket\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.633927 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.633858    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95f76fb-406d-4c59-a655-ae391d5952e3-istiod-ca-cert\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.633927 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.633877    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-envoy\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.634131 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.633958    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-socket\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.634131 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.633981    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-data\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.634131 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.634024    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzcsx\" (UniqueName: \"kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-kube-api-access-wzcsx\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735323 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735291    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-podinfo\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735476 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735341    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-credential-socket\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735476 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95f76fb-406d-4c59-a655-ae391d5952e3-istiod-ca-cert\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735476 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735404    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-envoy\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735476 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735464    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-socket\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735487    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-data\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735526    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzcsx\" (UniqueName: \"kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-kube-api-access-wzcsx\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735606    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-token\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735626    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-certs\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735911 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735818    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-credential-socket\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735911 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735848    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-socket\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.735911 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735898    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-data\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.736106 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.735997    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-certs\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.736436 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.736415    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95f76fb-406d-4c59-a655-ae391d5952e3-istiod-ca-cert\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.738133 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.738103    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-podinfo\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.738232 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.738138    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-envoy\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.745021 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.744985    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-token\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.745420 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.745403    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzcsx\" (UniqueName: \"kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-kube-api-access-wzcsx\") pod \"e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") " pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.815293 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.815233    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:54.965357 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.965332    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"]
Jul 17 14:18:54.966977 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:18:54.966953    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95f76fb_406d_4c59_a655_ae391d5952e3.slice/crio-b7113a6322dd8147bb8a06033e85113bf7f155d9cb47e89dea404ed4677d20d3 WatchSource:0}: Error finding container b7113a6322dd8147bb8a06033e85113bf7f155d9cb47e89dea404ed4677d20d3: Status 404 returned error can't find the container with id b7113a6322dd8147bb8a06033e85113bf7f155d9cb47e89dea404ed4677d20d3
Jul 17 14:18:54.969255 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.969214    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:18:54.969343 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.969312    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:18:54.969384 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:54.969357    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:18:55.880669 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:55.880627    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" event={"ID":"f95f76fb-406d-4c59-a655-ae391d5952e3","Type":"ContainerStarted","Data":"c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364"}
Jul 17 14:18:55.881157 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:55.880691    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" event={"ID":"f95f76fb-406d-4c59-a655-ae391d5952e3","Type":"ContainerStarted","Data":"b7113a6322dd8147bb8a06033e85113bf7f155d9cb47e89dea404ed4677d20d3"}
Jul 17 14:18:55.905876 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:55.905829    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" podStartSLOduration=1.905815069 podStartE2EDuration="1.905815069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:18:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:18:55.90340384 +0000 UTC m=+2021.609850677" watchObservedRunningTime="2026-07-17 14:18:55.905815069 +0000 UTC m=+2021.612261903"
Jul 17 14:18:56.816165 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:56.816132    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:56.821703 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:56.821675    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:56.886631 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:56.886600    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:18:56.887552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:18:56.887535    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:19:00.970899 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:00.970869    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"]
Jul 17 14:19:00.971326 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:00.971312    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy"
Jul 17 14:19:00.971326 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:00.971327    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="51797a43-483b-49e4-b2ad-66804cc6e532" containerName="istio-proxy"
Jul 17 14:19:00.971416 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:00.971407    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a69359ed-0fb8-44a2-95ef-561994dab94b" containerName="revoke-keys"
Jul 17 14:19:00.980062 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:00.980031    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"
Jul 17 14:19:00.983697 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:00.983673    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:19:00.983971 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:00.983952    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:19:00.984617 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:00.984598    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"]
Jul 17 14:19:01.098964 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:01.098926    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f17b74a-0b6c-406c-a99e-5b00b13308af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-5c25h\" (UID: \"9f17b74a-0b6c-406c-a99e-5b00b13308af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"
Jul 17 14:19:01.199790 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:01.199757    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f17b74a-0b6c-406c-a99e-5b00b13308af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-5c25h\" (UID: \"9f17b74a-0b6c-406c-a99e-5b00b13308af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"
Jul 17 14:19:01.200383 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:01.200364    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f17b74a-0b6c-406c-a99e-5b00b13308af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-5c25h\" (UID: \"9f17b74a-0b6c-406c-a99e-5b00b13308af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"
Jul 17 14:19:01.294974 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:01.294893    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"
Jul 17 14:19:01.490157 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:01.490132    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"]
Jul 17 14:19:01.493221 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:19:01.493192    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f17b74a_0b6c_406c_a99e_5b00b13308af.slice/crio-256581fa8a82c7d1db7c8704c5b59415b48f504dd9e8184d7b8a07b9b0ae6879 WatchSource:0}: Error finding container 256581fa8a82c7d1db7c8704c5b59415b48f504dd9e8184d7b8a07b9b0ae6879: Status 404 returned error can't find the container with id 256581fa8a82c7d1db7c8704c5b59415b48f504dd9e8184d7b8a07b9b0ae6879
Jul 17 14:19:01.917972 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:01.917939    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h" event={"ID":"9f17b74a-0b6c-406c-a99e-5b00b13308af","Type":"ContainerStarted","Data":"2987584df632e2c3c5a25e5ea1e027cb49b1f009a72cfa548ed0d8674f9605b1"}
Jul 17 14:19:01.918182 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:01.917993    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h" event={"ID":"9f17b74a-0b6c-406c-a99e-5b00b13308af","Type":"ContainerStarted","Data":"256581fa8a82c7d1db7c8704c5b59415b48f504dd9e8184d7b8a07b9b0ae6879"}
Jul 17 14:19:02.926495 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:02.926461    2561 generic.go:356] "Generic (PLEG): container finished" podID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerID="2987584df632e2c3c5a25e5ea1e027cb49b1f009a72cfa548ed0d8674f9605b1" exitCode=7
Jul 17 14:19:02.927122 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:02.926651    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h" event={"ID":"9f17b74a-0b6c-406c-a99e-5b00b13308af","Type":"ContainerDied","Data":"2987584df632e2c3c5a25e5ea1e027cb49b1f009a72cfa548ed0d8674f9605b1"}
Jul 17 14:19:02.927265 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:02.926740    2561 scope.go:117] "RemoveContainer" containerID="2987584df632e2c3c5a25e5ea1e027cb49b1f009a72cfa548ed0d8674f9605b1"
Jul 17 14:19:03.939409 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:03.938218    2561 generic.go:356] "Generic (PLEG): container finished" podID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerID="f06a8939c5a4d274e8ac02baae22bd84b48972ed518a86a38073f374eb632bab" exitCode=7
Jul 17 14:19:03.939409 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:03.938302    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h" event={"ID":"9f17b74a-0b6c-406c-a99e-5b00b13308af","Type":"ContainerDied","Data":"f06a8939c5a4d274e8ac02baae22bd84b48972ed518a86a38073f374eb632bab"}
Jul 17 14:19:03.939409 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:03.938356    2561 scope.go:117] "RemoveContainer" containerID="2987584df632e2c3c5a25e5ea1e027cb49b1f009a72cfa548ed0d8674f9605b1"
Jul 17 14:19:03.939409 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:03.938785    2561 scope.go:117] "RemoveContainer" containerID="f06a8939c5a4d274e8ac02baae22bd84b48972ed518a86a38073f374eb632bab"
Jul 17 14:19:03.939409 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:19:03.939027    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-5c25h_odh-ai-gateway-infra(9f17b74a-0b6c-406c-a99e-5b00b13308af)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af"
Jul 17 14:19:04.945359 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:04.945332    2561 scope.go:117] "RemoveContainer" containerID="f06a8939c5a4d274e8ac02baae22bd84b48972ed518a86a38073f374eb632bab"
Jul 17 14:19:04.945699 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:19:04.945587    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-5c25h_odh-ai-gateway-infra(9f17b74a-0b6c-406c-a99e-5b00b13308af)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af"
Jul 17 14:19:07.963715 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:07.963680    2561 generic.go:356] "Generic (PLEG): container finished" podID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerID="c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364" exitCode=0
Jul 17 14:19:07.964170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:07.963748    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" event={"ID":"f95f76fb-406d-4c59-a655-ae391d5952e3","Type":"ContainerDied","Data":"c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364"}
Jul 17 14:19:07.964242 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:07.964228    2561 scope.go:117] "RemoveContainer" containerID="c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364"
Jul 17 14:19:07.965326 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:07.965296    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:19:07.965439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:07.965381    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:19:07.965439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:07.965429    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:19:08.971207 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:08.971168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" event={"ID":"f95f76fb-406d-4c59-a655-ae391d5952e3","Type":"ContainerStarted","Data":"4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5"}
Jul 17 14:19:09.815564 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:09.815527    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:19:09.816808 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:09.816781    2561 patch_prober.go:28] interesting pod/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 17 14:19:09.816912 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:09.816830    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 17 14:19:10.815640 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:10.815610    2561 patch_prober.go:28] interesting pod/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 17 14:19:10.816065 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:10.815669    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 17 14:19:11.815961 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:11.815926    2561 patch_prober.go:28] interesting pod/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 17 14:19:11.816400 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:11.815981    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 17 14:19:11.886976 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:11.886938    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:19:12.815807 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:12.815769    2561 patch_prober.go:28] interesting pod/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 17 14:19:12.815975 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:12.815827    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 17 14:19:13.820256 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:13.820225    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:19:13.821128 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:13.821106    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:19:18.766661 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:18.766628    2561 scope.go:117] "RemoveContainer" containerID="f06a8939c5a4d274e8ac02baae22bd84b48972ed518a86a38073f374eb632bab"
Jul 17 14:19:20.031864 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:20.031829    2561 generic.go:356] "Generic (PLEG): container finished" podID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerID="254157305c777c2cfe11c85760f52a660d74c2115b826ba409c55c06e240796b" exitCode=7
Jul 17 14:19:20.032367 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:20.031897    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h" event={"ID":"9f17b74a-0b6c-406c-a99e-5b00b13308af","Type":"ContainerDied","Data":"254157305c777c2cfe11c85760f52a660d74c2115b826ba409c55c06e240796b"}
Jul 17 14:19:20.032367 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:20.031944    2561 scope.go:117] "RemoveContainer" containerID="f06a8939c5a4d274e8ac02baae22bd84b48972ed518a86a38073f374eb632bab"
Jul 17 14:19:20.032367 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:20.032274    2561 scope.go:117] "RemoveContainer" containerID="254157305c777c2cfe11c85760f52a660d74c2115b826ba409c55c06e240796b"
Jul 17 14:19:20.032562 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:19:20.032540    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-5c25h_odh-ai-gateway-infra(9f17b74a-0b6c-406c-a99e-5b00b13308af)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af"
Jul 17 14:19:21.061740 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.061700    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"]
Jul 17 14:19:21.211959 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.211937    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"
Jul 17 14:19:21.405425 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.405336    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f17b74a-0b6c-406c-a99e-5b00b13308af-maas-api-service-ca\") pod \"9f17b74a-0b6c-406c-a99e-5b00b13308af\" (UID: \"9f17b74a-0b6c-406c-a99e-5b00b13308af\") "
Jul 17 14:19:21.405773 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.405744    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f17b74a-0b6c-406c-a99e-5b00b13308af-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9f17b74a-0b6c-406c-a99e-5b00b13308af" (UID: "9f17b74a-0b6c-406c-a99e-5b00b13308af"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:19:21.506369 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.506338    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f17b74a-0b6c-406c-a99e-5b00b13308af-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:19:21.752364 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.752330    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"]
Jul 17 14:19:21.752769 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.752757    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerName="revoke-keys"
Jul 17 14:19:21.752814 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.752770    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerName="revoke-keys"
Jul 17 14:19:21.752814 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.752795    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerName="revoke-keys"
Jul 17 14:19:21.752814 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.752802    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerName="revoke-keys"
Jul 17 14:19:21.752937 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.752870    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerName="revoke-keys"
Jul 17 14:19:21.752937 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.752882    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerName="revoke-keys"
Jul 17 14:19:21.755792 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.755775    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"
Jul 17 14:19:21.764084 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.764061    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"]
Jul 17 14:19:21.910232 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:21.910194    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d473457f-41d1-463a-9c2e-14af8324540b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl\" (UID: \"d473457f-41d1-463a-9c2e-14af8324540b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"
Jul 17 14:19:22.010913 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:22.010821    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d473457f-41d1-463a-9c2e-14af8324540b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl\" (UID: \"d473457f-41d1-463a-9c2e-14af8324540b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"
Jul 17 14:19:22.011581 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:22.011555    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d473457f-41d1-463a-9c2e-14af8324540b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl\" (UID: \"d473457f-41d1-463a-9c2e-14af8324540b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"
Jul 17 14:19:22.048879 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:22.048852    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"
Jul 17 14:19:22.049104 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:22.048877    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h" event={"ID":"9f17b74a-0b6c-406c-a99e-5b00b13308af","Type":"ContainerDied","Data":"256581fa8a82c7d1db7c8704c5b59415b48f504dd9e8184d7b8a07b9b0ae6879"}
Jul 17 14:19:22.049104 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:22.048935    2561 scope.go:117] "RemoveContainer" containerID="254157305c777c2cfe11c85760f52a660d74c2115b826ba409c55c06e240796b"
Jul 17 14:19:22.067650 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:22.067628    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"
Jul 17 14:19:22.072723 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:22.072703    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"]
Jul 17 14:19:22.077681 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:22.077659    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-5c25h"]
Jul 17 14:19:22.213096 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:22.213057    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"]
Jul 17 14:19:22.214578 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:19:22.214549    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd473457f_41d1_463a_9c2e_14af8324540b.slice/crio-a660a5ec1d632065ff66e73fe755509e7818d85352009a927ad8ea443d461b75 WatchSource:0}: Error finding container a660a5ec1d632065ff66e73fe755509e7818d85352009a927ad8ea443d461b75: Status 404 returned error can't find the container with id a660a5ec1d632065ff66e73fe755509e7818d85352009a927ad8ea443d461b75
Jul 17 14:19:22.777924 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:22.777806    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af" path="/var/lib/kubelet/pods/9f17b74a-0b6c-406c-a99e-5b00b13308af/volumes"
Jul 17 14:19:23.055766 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:23.055679    2561 generic.go:356] "Generic (PLEG): container finished" podID="d473457f-41d1-463a-9c2e-14af8324540b" containerID="16783ab6edd6d353b558352d61ae55f554b67bd0146989d59268163a0a1e06cc" exitCode=6
Jul 17 14:19:23.055927 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:23.055773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl" event={"ID":"d473457f-41d1-463a-9c2e-14af8324540b","Type":"ContainerDied","Data":"16783ab6edd6d353b558352d61ae55f554b67bd0146989d59268163a0a1e06cc"}
Jul 17 14:19:23.055927 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:23.055834    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl" event={"ID":"d473457f-41d1-463a-9c2e-14af8324540b","Type":"ContainerStarted","Data":"a660a5ec1d632065ff66e73fe755509e7818d85352009a927ad8ea443d461b75"}
Jul 17 14:19:23.056034 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:23.055959    2561 scope.go:117] "RemoveContainer" containerID="16783ab6edd6d353b558352d61ae55f554b67bd0146989d59268163a0a1e06cc"
Jul 17 14:19:24.064001 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:24.063969    2561 generic.go:356] "Generic (PLEG): container finished" podID="d473457f-41d1-463a-9c2e-14af8324540b" containerID="09fa6453ec5927142e814b15bbf6141d897f2bfb3a11b2be2b8f7cefcd25b1fc" exitCode=6
Jul 17 14:19:24.064526 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:24.064039    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl" event={"ID":"d473457f-41d1-463a-9c2e-14af8324540b","Type":"ContainerDied","Data":"09fa6453ec5927142e814b15bbf6141d897f2bfb3a11b2be2b8f7cefcd25b1fc"}
Jul 17 14:19:24.064526 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:24.064101    2561 scope.go:117] "RemoveContainer" containerID="16783ab6edd6d353b558352d61ae55f554b67bd0146989d59268163a0a1e06cc"
Jul 17 14:19:24.064526 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:24.064351    2561 scope.go:117] "RemoveContainer" containerID="09fa6453ec5927142e814b15bbf6141d897f2bfb3a11b2be2b8f7cefcd25b1fc"
Jul 17 14:19:24.064682 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:19:24.064611    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl_odh-ai-gateway-infra(d473457f-41d1-463a-9c2e-14af8324540b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl" podUID="d473457f-41d1-463a-9c2e-14af8324540b"
Jul 17 14:19:25.071040 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:25.070991    2561 scope.go:117] "RemoveContainer" containerID="09fa6453ec5927142e814b15bbf6141d897f2bfb3a11b2be2b8f7cefcd25b1fc"
Jul 17 14:19:25.071460 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:19:25.071256    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl_odh-ai-gateway-infra(d473457f-41d1-463a-9c2e-14af8324540b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl" podUID="d473457f-41d1-463a-9c2e-14af8324540b"
Jul 17 14:19:36.772996 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:36.772925    2561 scope.go:117] "RemoveContainer" containerID="09fa6453ec5927142e814b15bbf6141d897f2bfb3a11b2be2b8f7cefcd25b1fc"
Jul 17 14:19:37.138256 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:37.138219    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl" event={"ID":"d473457f-41d1-463a-9c2e-14af8324540b","Type":"ContainerStarted","Data":"fe4ea4b3c72e89a1e0e14fdae2df46b7a9d03c0ccb59594570db58456766c092"}
Jul 17 14:19:38.145241 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:38.145207    2561 generic.go:356] "Generic (PLEG): container finished" podID="d473457f-41d1-463a-9c2e-14af8324540b" containerID="fe4ea4b3c72e89a1e0e14fdae2df46b7a9d03c0ccb59594570db58456766c092" exitCode=6
Jul 17 14:19:38.145650 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:38.145283    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl" event={"ID":"d473457f-41d1-463a-9c2e-14af8324540b","Type":"ContainerDied","Data":"fe4ea4b3c72e89a1e0e14fdae2df46b7a9d03c0ccb59594570db58456766c092"}
Jul 17 14:19:38.145650 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:38.145345    2561 scope.go:117] "RemoveContainer" containerID="09fa6453ec5927142e814b15bbf6141d897f2bfb3a11b2be2b8f7cefcd25b1fc"
Jul 17 14:19:38.145650 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:38.145648    2561 scope.go:117] "RemoveContainer" containerID="fe4ea4b3c72e89a1e0e14fdae2df46b7a9d03c0ccb59594570db58456766c092"
Jul 17 14:19:38.145900 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:19:38.145878    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl_odh-ai-gateway-infra(d473457f-41d1-463a-9c2e-14af8324540b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl" podUID="d473457f-41d1-463a-9c2e-14af8324540b"
Jul 17 14:19:38.171452 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:38.171425    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"]
Jul 17 14:19:39.297916 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:39.297892    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"
Jul 17 14:19:39.375121 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:39.375090    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d473457f-41d1-463a-9c2e-14af8324540b-maas-api-service-ca\") pod \"d473457f-41d1-463a-9c2e-14af8324540b\" (UID: \"d473457f-41d1-463a-9c2e-14af8324540b\") "
Jul 17 14:19:39.375428 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:39.375405    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d473457f-41d1-463a-9c2e-14af8324540b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d473457f-41d1-463a-9c2e-14af8324540b" (UID: "d473457f-41d1-463a-9c2e-14af8324540b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:19:39.475894 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:39.475855    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d473457f-41d1-463a-9c2e-14af8324540b-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:19:40.158102 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:40.158051    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl" event={"ID":"d473457f-41d1-463a-9c2e-14af8324540b","Type":"ContainerDied","Data":"a660a5ec1d632065ff66e73fe755509e7818d85352009a927ad8ea443d461b75"}
Jul 17 14:19:40.158102 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:40.158099    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"
Jul 17 14:19:40.158334 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:40.158171    2561 scope.go:117] "RemoveContainer" containerID="fe4ea4b3c72e89a1e0e14fdae2df46b7a9d03c0ccb59594570db58456766c092"
Jul 17 14:19:40.184361 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:40.184330    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"]
Jul 17 14:19:40.195146 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:40.195113    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-vnrpl"]
Jul 17 14:19:40.773085 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:19:40.773051    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d473457f-41d1-463a-9c2e-14af8324540b" path="/var/lib/kubelet/pods/d473457f-41d1-463a-9c2e-14af8324540b/volumes"
Jul 17 14:20:11.796990 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.796944    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"]
Jul 17 14:20:11.797483 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.797369    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d473457f-41d1-463a-9c2e-14af8324540b" containerName="revoke-keys"
Jul 17 14:20:11.797483 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.797380    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d473457f-41d1-463a-9c2e-14af8324540b" containerName="revoke-keys"
Jul 17 14:20:11.797483 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.797392    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d473457f-41d1-463a-9c2e-14af8324540b" containerName="revoke-keys"
Jul 17 14:20:11.797483 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.797397    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d473457f-41d1-463a-9c2e-14af8324540b" containerName="revoke-keys"
Jul 17 14:20:11.797483 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.797417    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerName="revoke-keys"
Jul 17 14:20:11.797483 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.797423    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerName="revoke-keys"
Jul 17 14:20:11.797674 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.797490    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f17b74a-0b6c-406c-a99e-5b00b13308af" containerName="revoke-keys"
Jul 17 14:20:11.797674 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.797496    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d473457f-41d1-463a-9c2e-14af8324540b" containerName="revoke-keys"
Jul 17 14:20:11.797674 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.797502    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d473457f-41d1-463a-9c2e-14af8324540b" containerName="revoke-keys"
Jul 17 14:20:11.797674 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.797509    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d473457f-41d1-463a-9c2e-14af8324540b" containerName="revoke-keys"
Jul 17 14:20:11.800648 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.800631    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"
Jul 17 14:20:11.803278 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.803249    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:20:11.803388 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.803358    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:20:11.808181 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.808117    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"]
Jul 17 14:20:11.878353 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.878325    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93bfb1e1-3ea3-4269-ba9b-7778e66caf5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9\" (UID: \"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"
Jul 17 14:20:11.979159 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.979132    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93bfb1e1-3ea3-4269-ba9b-7778e66caf5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9\" (UID: \"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"
Jul 17 14:20:11.979750 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:11.979727    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93bfb1e1-3ea3-4269-ba9b-7778e66caf5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9\" (UID: \"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"
Jul 17 14:20:12.113141 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:12.113064    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"
Jul 17 14:20:12.248844 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:12.248821    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"]
Jul 17 14:20:12.250653 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:20:12.250626    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93bfb1e1_3ea3_4269_ba9b_7778e66caf5b.slice/crio-874afc1cd5282041e3327a7434c1487dcbbf9099e51bf57ce5e7a21327ae92aa WatchSource:0}: Error finding container 874afc1cd5282041e3327a7434c1487dcbbf9099e51bf57ce5e7a21327ae92aa: Status 404 returned error can't find the container with id 874afc1cd5282041e3327a7434c1487dcbbf9099e51bf57ce5e7a21327ae92aa
Jul 17 14:20:12.330948 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:12.330919    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9" event={"ID":"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b","Type":"ContainerStarted","Data":"874afc1cd5282041e3327a7434c1487dcbbf9099e51bf57ce5e7a21327ae92aa"}
Jul 17 14:20:13.338605 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:13.338509    2561 generic.go:356] "Generic (PLEG): container finished" podID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerID="91fd50bcd7f768879c850fdfe541d83cd2456a2744c5a630b92a2b8da92254fc" exitCode=6
Jul 17 14:20:13.338605 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:13.338585    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9" event={"ID":"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b","Type":"ContainerDied","Data":"91fd50bcd7f768879c850fdfe541d83cd2456a2744c5a630b92a2b8da92254fc"}
Jul 17 14:20:13.339088 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:13.338768    2561 scope.go:117] "RemoveContainer" containerID="91fd50bcd7f768879c850fdfe541d83cd2456a2744c5a630b92a2b8da92254fc"
Jul 17 14:20:14.346434 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:14.346349    2561 generic.go:356] "Generic (PLEG): container finished" podID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerID="325b05ee682d73e759e91933950c2fded2cb8c775bcbade4a2cd13a2e3e0fa20" exitCode=6
Jul 17 14:20:14.346434 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:14.346393    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9" event={"ID":"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b","Type":"ContainerDied","Data":"325b05ee682d73e759e91933950c2fded2cb8c775bcbade4a2cd13a2e3e0fa20"}
Jul 17 14:20:14.346919 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:14.346458    2561 scope.go:117] "RemoveContainer" containerID="91fd50bcd7f768879c850fdfe541d83cd2456a2744c5a630b92a2b8da92254fc"
Jul 17 14:20:14.346919 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:14.346625    2561 scope.go:117] "RemoveContainer" containerID="325b05ee682d73e759e91933950c2fded2cb8c775bcbade4a2cd13a2e3e0fa20"
Jul 17 14:20:14.346919 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:20:14.346868    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9_odh-ai-gateway-infra(93bfb1e1-3ea3-4269-ba9b-7778e66caf5b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b"
Jul 17 14:20:15.352741 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:15.352666    2561 scope.go:117] "RemoveContainer" containerID="325b05ee682d73e759e91933950c2fded2cb8c775bcbade4a2cd13a2e3e0fa20"
Jul 17 14:20:15.353112 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:20:15.352852    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9_odh-ai-gateway-infra(93bfb1e1-3ea3-4269-ba9b-7778e66caf5b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b"
Jul 17 14:20:30.766526 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:30.766499    2561 scope.go:117] "RemoveContainer" containerID="325b05ee682d73e759e91933950c2fded2cb8c775bcbade4a2cd13a2e3e0fa20"
Jul 17 14:20:31.440061 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:31.440036    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9" event={"ID":"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b","Type":"ContainerStarted","Data":"3051e4b0038c53323d307d7e17076c7c314c64d17e1d91d1b6137a7e5d20cef1"}
Jul 17 14:20:31.440371 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:31.440351    2561 scope.go:117] "RemoveContainer" containerID="3051e4b0038c53323d307d7e17076c7c314c64d17e1d91d1b6137a7e5d20cef1"
Jul 17 14:20:31.440566 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:20:31.440550    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9_odh-ai-gateway-infra(93bfb1e1-3ea3-4269-ba9b-7778e66caf5b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b"
Jul 17 14:20:32.448108 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:32.448079    2561 generic.go:356] "Generic (PLEG): container finished" podID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerID="3051e4b0038c53323d307d7e17076c7c314c64d17e1d91d1b6137a7e5d20cef1" exitCode=6
Jul 17 14:20:32.448581 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:32.448158    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9" event={"ID":"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b","Type":"ContainerDied","Data":"3051e4b0038c53323d307d7e17076c7c314c64d17e1d91d1b6137a7e5d20cef1"}
Jul 17 14:20:32.448581 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:32.448238    2561 scope.go:117] "RemoveContainer" containerID="325b05ee682d73e759e91933950c2fded2cb8c775bcbade4a2cd13a2e3e0fa20"
Jul 17 14:20:32.448700 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:32.448649    2561 scope.go:117] "RemoveContainer" containerID="3051e4b0038c53323d307d7e17076c7c314c64d17e1d91d1b6137a7e5d20cef1"
Jul 17 14:20:32.448872 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:20:32.448850    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9_odh-ai-gateway-infra(93bfb1e1-3ea3-4269-ba9b-7778e66caf5b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b"
Jul 17 14:20:32.467500 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:32.467476    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"]
Jul 17 14:20:33.601849 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:33.601827    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"
Jul 17 14:20:33.667331 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:33.667305    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93bfb1e1-3ea3-4269-ba9b-7778e66caf5b-maas-api-service-ca\") pod \"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b\" (UID: \"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b\") "
Jul 17 14:20:33.667629 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:33.667605    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93bfb1e1-3ea3-4269-ba9b-7778e66caf5b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" (UID: "93bfb1e1-3ea3-4269-ba9b-7778e66caf5b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:20:33.768869 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:33.768793    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93bfb1e1-3ea3-4269-ba9b-7778e66caf5b-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:20:34.461558 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:34.461529    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"
Jul 17 14:20:34.461737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:34.461559    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9" event={"ID":"93bfb1e1-3ea3-4269-ba9b-7778e66caf5b","Type":"ContainerDied","Data":"874afc1cd5282041e3327a7434c1487dcbbf9099e51bf57ce5e7a21327ae92aa"}
Jul 17 14:20:34.461737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:34.461621    2561 scope.go:117] "RemoveContainer" containerID="3051e4b0038c53323d307d7e17076c7c314c64d17e1d91d1b6137a7e5d20cef1"
Jul 17 14:20:34.496194 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:34.496163    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"]
Jul 17 14:20:34.496946 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:34.496917    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-qngv9"]
Jul 17 14:20:34.772103 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:34.772025    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" path="/var/lib/kubelet/pods/93bfb1e1-3ea3-4269-ba9b-7778e66caf5b/volumes"
Jul 17 14:20:52.745536 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.745503    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"]
Jul 17 14:20:52.746045 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.745933    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d473457f-41d1-463a-9c2e-14af8324540b" containerName="revoke-keys"
Jul 17 14:20:52.746045 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.745944    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d473457f-41d1-463a-9c2e-14af8324540b" containerName="revoke-keys"
Jul 17 14:20:52.746045 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.745956    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerName="revoke-keys"
Jul 17 14:20:52.746045 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.745961    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerName="revoke-keys"
Jul 17 14:20:52.746045 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.745977    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerName="revoke-keys"
Jul 17 14:20:52.746045 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.745982    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerName="revoke-keys"
Jul 17 14:20:52.746254 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.746072    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerName="revoke-keys"
Jul 17 14:20:52.746254 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.746080    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerName="revoke-keys"
Jul 17 14:20:52.746254 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.746088    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerName="revoke-keys"
Jul 17 14:20:52.749214 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.749198    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"
Jul 17 14:20:52.751719 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.751696    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:20:52.751859 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.751699    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:20:52.757047 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.756717    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"]
Jul 17 14:20:52.843761 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.843732    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/286d316b-41ab-4275-a713-bc397bdd01b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7\" (UID: \"286d316b-41ab-4275-a713-bc397bdd01b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"
Jul 17 14:20:52.945141 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.945114    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/286d316b-41ab-4275-a713-bc397bdd01b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7\" (UID: \"286d316b-41ab-4275-a713-bc397bdd01b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"
Jul 17 14:20:52.945725 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:52.945700    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/286d316b-41ab-4275-a713-bc397bdd01b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7\" (UID: \"286d316b-41ab-4275-a713-bc397bdd01b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"
Jul 17 14:20:53.061055 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:53.060974    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"
Jul 17 14:20:53.194989 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:53.194825    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"]
Jul 17 14:20:53.197361 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:20:53.197331    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286d316b_41ab_4275_a713_bc397bdd01b8.slice/crio-1f1703eb03cf49ecdcd7b9aa1703b0ceaee155d1ab16eb6043162f73c67f136b WatchSource:0}: Error finding container 1f1703eb03cf49ecdcd7b9aa1703b0ceaee155d1ab16eb6043162f73c67f136b: Status 404 returned error can't find the container with id 1f1703eb03cf49ecdcd7b9aa1703b0ceaee155d1ab16eb6043162f73c67f136b
Jul 17 14:20:53.573835 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:53.573801    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7" event={"ID":"286d316b-41ab-4275-a713-bc397bdd01b8","Type":"ContainerStarted","Data":"073ca76c4fb797862e8a241cb7bc568b99c176e7f77011c78e272b6f6201de0c"}
Jul 17 14:20:53.573941 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:53.573850    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7" event={"ID":"286d316b-41ab-4275-a713-bc397bdd01b8","Type":"ContainerStarted","Data":"1f1703eb03cf49ecdcd7b9aa1703b0ceaee155d1ab16eb6043162f73c67f136b"}
Jul 17 14:20:53.591640 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:53.591599    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7" podStartSLOduration=1.591584808 podStartE2EDuration="1.591584808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:20:53.590625725 +0000 UTC m=+2139.297072559" watchObservedRunningTime="2026-07-17 14:20:53.591584808 +0000 UTC m=+2139.298031641"
Jul 17 14:20:54.580897 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:54.580864    2561 generic.go:356] "Generic (PLEG): container finished" podID="286d316b-41ab-4275-a713-bc397bdd01b8" containerID="073ca76c4fb797862e8a241cb7bc568b99c176e7f77011c78e272b6f6201de0c" exitCode=6
Jul 17 14:20:54.581319 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:54.580947    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7" event={"ID":"286d316b-41ab-4275-a713-bc397bdd01b8","Type":"ContainerDied","Data":"073ca76c4fb797862e8a241cb7bc568b99c176e7f77011c78e272b6f6201de0c"}
Jul 17 14:20:54.581319 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:54.581126    2561 scope.go:117] "RemoveContainer" containerID="073ca76c4fb797862e8a241cb7bc568b99c176e7f77011c78e272b6f6201de0c"
Jul 17 14:20:55.588099 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:55.588065    2561 generic.go:356] "Generic (PLEG): container finished" podID="286d316b-41ab-4275-a713-bc397bdd01b8" containerID="acd972bcc848d5790d7986f65dd5863f4b1b032e7ea2d1aedbc8d5f91ca4d6fb" exitCode=6
Jul 17 14:20:55.588531 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:55.588149    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7" event={"ID":"286d316b-41ab-4275-a713-bc397bdd01b8","Type":"ContainerDied","Data":"acd972bcc848d5790d7986f65dd5863f4b1b032e7ea2d1aedbc8d5f91ca4d6fb"}
Jul 17 14:20:55.588531 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:55.588210    2561 scope.go:117] "RemoveContainer" containerID="073ca76c4fb797862e8a241cb7bc568b99c176e7f77011c78e272b6f6201de0c"
Jul 17 14:20:55.588531 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:55.588458    2561 scope.go:117] "RemoveContainer" containerID="acd972bcc848d5790d7986f65dd5863f4b1b032e7ea2d1aedbc8d5f91ca4d6fb"
Jul 17 14:20:55.588715 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:20:55.588653    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7_odh-ai-gateway-infra(286d316b-41ab-4275-a713-bc397bdd01b8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7" podUID="286d316b-41ab-4275-a713-bc397bdd01b8"
Jul 17 14:20:56.595307 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:20:56.595275    2561 scope.go:117] "RemoveContainer" containerID="acd972bcc848d5790d7986f65dd5863f4b1b032e7ea2d1aedbc8d5f91ca4d6fb"
Jul 17 14:20:56.595673 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:20:56.595503    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7_odh-ai-gateway-infra(286d316b-41ab-4275-a713-bc397bdd01b8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7" podUID="286d316b-41ab-4275-a713-bc397bdd01b8"
Jul 17 14:21:01.066316 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.066286    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"]
Jul 17 14:21:01.066745 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.066700    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerName="revoke-keys"
Jul 17 14:21:01.066745 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.066710    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="93bfb1e1-3ea3-4269-ba9b-7778e66caf5b" containerName="revoke-keys"
Jul 17 14:21:01.071054 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.071038    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"
Jul 17 14:21:01.079864 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.079842    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"]
Jul 17 14:21:01.122087 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.122051    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36898e2b-fd0a-45b3-a296-6ba170589e41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-7vrbd\" (UID: \"36898e2b-fd0a-45b3-a296-6ba170589e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"
Jul 17 14:21:01.222938 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.222907    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36898e2b-fd0a-45b3-a296-6ba170589e41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-7vrbd\" (UID: \"36898e2b-fd0a-45b3-a296-6ba170589e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"
Jul 17 14:21:01.223529 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.223510    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36898e2b-fd0a-45b3-a296-6ba170589e41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-7vrbd\" (UID: \"36898e2b-fd0a-45b3-a296-6ba170589e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"
Jul 17 14:21:01.382235 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.382142    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"
Jul 17 14:21:01.517920 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.517888    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"]
Jul 17 14:21:01.519810 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:21:01.519780    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36898e2b_fd0a_45b3_a296_6ba170589e41.slice/crio-f52259921b3145838f8f8281cf3b8e8395f579eb4585984929426c46a1cfbbb4 WatchSource:0}: Error finding container f52259921b3145838f8f8281cf3b8e8395f579eb4585984929426c46a1cfbbb4: Status 404 returned error can't find the container with id f52259921b3145838f8f8281cf3b8e8395f579eb4585984929426c46a1cfbbb4
Jul 17 14:21:01.625875 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:01.625846    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd" event={"ID":"36898e2b-fd0a-45b3-a296-6ba170589e41","Type":"ContainerStarted","Data":"f52259921b3145838f8f8281cf3b8e8395f579eb4585984929426c46a1cfbbb4"}
Jul 17 14:21:02.632729 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:02.632688    2561 generic.go:356] "Generic (PLEG): container finished" podID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerID="f28f596bf67b93515219660e99b1e41adafe01673c37dd62141bc3f40bede2d0" exitCode=7
Jul 17 14:21:02.633151 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:02.632776    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd" event={"ID":"36898e2b-fd0a-45b3-a296-6ba170589e41","Type":"ContainerDied","Data":"f28f596bf67b93515219660e99b1e41adafe01673c37dd62141bc3f40bede2d0"}
Jul 17 14:21:02.633151 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:02.632951    2561 scope.go:117] "RemoveContainer" containerID="f28f596bf67b93515219660e99b1e41adafe01673c37dd62141bc3f40bede2d0"
Jul 17 14:21:03.639917 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:03.639826    2561 generic.go:356] "Generic (PLEG): container finished" podID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerID="56499abeb2f1ca66866173e4e90bd5eec82103143e18d20283a7cec6a712f104" exitCode=7
Jul 17 14:21:03.639917 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:03.639888    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd" event={"ID":"36898e2b-fd0a-45b3-a296-6ba170589e41","Type":"ContainerDied","Data":"56499abeb2f1ca66866173e4e90bd5eec82103143e18d20283a7cec6a712f104"}
Jul 17 14:21:03.640397 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:03.639955    2561 scope.go:117] "RemoveContainer" containerID="f28f596bf67b93515219660e99b1e41adafe01673c37dd62141bc3f40bede2d0"
Jul 17 14:21:03.640397 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:03.640230    2561 scope.go:117] "RemoveContainer" containerID="56499abeb2f1ca66866173e4e90bd5eec82103143e18d20283a7cec6a712f104"
Jul 17 14:21:03.640506 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:21:03.640485    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-7vrbd_odh-ai-gateway-infra(36898e2b-fd0a-45b3-a296-6ba170589e41)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41"
Jul 17 14:21:04.646758 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:04.646729    2561 scope.go:117] "RemoveContainer" containerID="56499abeb2f1ca66866173e4e90bd5eec82103143e18d20283a7cec6a712f104"
Jul 17 14:21:04.647152 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:21:04.646964    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-7vrbd_odh-ai-gateway-infra(36898e2b-fd0a-45b3-a296-6ba170589e41)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41"
Jul 17 14:21:09.766267 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:09.766197    2561 scope.go:117] "RemoveContainer" containerID="acd972bcc848d5790d7986f65dd5863f4b1b032e7ea2d1aedbc8d5f91ca4d6fb"
Jul 17 14:21:10.680799 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:10.680770    2561 generic.go:356] "Generic (PLEG): container finished" podID="286d316b-41ab-4275-a713-bc397bdd01b8" containerID="36299df45f6067c32d690522650bc8997c0046dc3eb9137ccdfdfb6dca3e6b1d" exitCode=6
Jul 17 14:21:10.680962 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:10.680846    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7" event={"ID":"286d316b-41ab-4275-a713-bc397bdd01b8","Type":"ContainerDied","Data":"36299df45f6067c32d690522650bc8997c0046dc3eb9137ccdfdfb6dca3e6b1d"}
Jul 17 14:21:10.680962 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:10.680911    2561 scope.go:117] "RemoveContainer" containerID="acd972bcc848d5790d7986f65dd5863f4b1b032e7ea2d1aedbc8d5f91ca4d6fb"
Jul 17 14:21:10.681236 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:10.681217    2561 scope.go:117] "RemoveContainer" containerID="36299df45f6067c32d690522650bc8997c0046dc3eb9137ccdfdfb6dca3e6b1d"
Jul 17 14:21:10.681501 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:21:10.681480    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7_odh-ai-gateway-infra(286d316b-41ab-4275-a713-bc397bdd01b8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7" podUID="286d316b-41ab-4275-a713-bc397bdd01b8"
Jul 17 14:21:11.709943 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:11.709906    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"]
Jul 17 14:21:11.841947 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:11.841918    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"]
Jul 17 14:21:11.847079 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:11.847053    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"
Jul 17 14:21:11.852981 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:11.852959    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"]
Jul 17 14:21:11.853487 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:11.853463    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"
Jul 17 14:21:11.922365 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:11.922336    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/286d316b-41ab-4275-a713-bc397bdd01b8-maas-api-service-ca\") pod \"286d316b-41ab-4275-a713-bc397bdd01b8\" (UID: \"286d316b-41ab-4275-a713-bc397bdd01b8\") "
Jul 17 14:21:11.922523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:11.922454    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a91270dc-3224-41a1-b837-7dbaa0124da4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-xr622\" (UID: \"a91270dc-3224-41a1-b837-7dbaa0124da4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"
Jul 17 14:21:11.922677 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:11.922655    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/286d316b-41ab-4275-a713-bc397bdd01b8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "286d316b-41ab-4275-a713-bc397bdd01b8" (UID: "286d316b-41ab-4275-a713-bc397bdd01b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:21:12.023407 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.023308    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a91270dc-3224-41a1-b837-7dbaa0124da4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-xr622\" (UID: \"a91270dc-3224-41a1-b837-7dbaa0124da4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"
Jul 17 14:21:12.023407 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.023388    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/286d316b-41ab-4275-a713-bc397bdd01b8-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:21:12.023914 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.023894    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a91270dc-3224-41a1-b837-7dbaa0124da4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-xr622\" (UID: \"a91270dc-3224-41a1-b837-7dbaa0124da4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"
Jul 17 14:21:12.162490 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.162458    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"
Jul 17 14:21:12.303873 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.303844    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"]
Jul 17 14:21:12.304374 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:21:12.304349    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91270dc_3224_41a1_b837_7dbaa0124da4.slice/crio-c4ba4ed026e4d44cd25994df2d9c72fc9fb4d334457c36ec289ba914501bdeb7 WatchSource:0}: Error finding container c4ba4ed026e4d44cd25994df2d9c72fc9fb4d334457c36ec289ba914501bdeb7: Status 404 returned error can't find the container with id c4ba4ed026e4d44cd25994df2d9c72fc9fb4d334457c36ec289ba914501bdeb7
Jul 17 14:21:12.695122 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.695089    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622" event={"ID":"a91270dc-3224-41a1-b837-7dbaa0124da4","Type":"ContainerStarted","Data":"06fa1b0768becac2716a7f49a9aeed4f4bba5ae1476307f09049fc3ac6c5d7ab"}
Jul 17 14:21:12.695320 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.695143    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622" event={"ID":"a91270dc-3224-41a1-b837-7dbaa0124da4","Type":"ContainerStarted","Data":"c4ba4ed026e4d44cd25994df2d9c72fc9fb4d334457c36ec289ba914501bdeb7"}
Jul 17 14:21:12.696428 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.696403    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"
Jul 17 14:21:12.696522 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.696427    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7" event={"ID":"286d316b-41ab-4275-a713-bc397bdd01b8","Type":"ContainerDied","Data":"1f1703eb03cf49ecdcd7b9aa1703b0ceaee155d1ab16eb6043162f73c67f136b"}
Jul 17 14:21:12.696522 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.696472    2561 scope.go:117] "RemoveContainer" containerID="36299df45f6067c32d690522650bc8997c0046dc3eb9137ccdfdfb6dca3e6b1d"
Jul 17 14:21:12.713116 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.713077    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622" podStartSLOduration=1.7130623539999998 podStartE2EDuration="1.713062354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:21:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:21:12.710082946 +0000 UTC m=+2158.416529796" watchObservedRunningTime="2026-07-17 14:21:12.713062354 +0000 UTC m=+2158.419509187"
Jul 17 14:21:12.728206 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.728175    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"]
Jul 17 14:21:12.733663 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.733634    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-4zhj7"]
Jul 17 14:21:12.777465 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:12.777432    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="286d316b-41ab-4275-a713-bc397bdd01b8" path="/var/lib/kubelet/pods/286d316b-41ab-4275-a713-bc397bdd01b8/volumes"
Jul 17 14:21:13.703351 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:13.703315    2561 generic.go:356] "Generic (PLEG): container finished" podID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerID="06fa1b0768becac2716a7f49a9aeed4f4bba5ae1476307f09049fc3ac6c5d7ab" exitCode=6
Jul 17 14:21:13.703543 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:13.703400    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622" event={"ID":"a91270dc-3224-41a1-b837-7dbaa0124da4","Type":"ContainerDied","Data":"06fa1b0768becac2716a7f49a9aeed4f4bba5ae1476307f09049fc3ac6c5d7ab"}
Jul 17 14:21:13.703650 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:13.703625    2561 scope.go:117] "RemoveContainer" containerID="06fa1b0768becac2716a7f49a9aeed4f4bba5ae1476307f09049fc3ac6c5d7ab"
Jul 17 14:21:14.712428 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:14.712352    2561 generic.go:356] "Generic (PLEG): container finished" podID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerID="4aa725a475561f340fc38c90ab1edfcf76910f9d795c87c4c329c705ff44ed4a" exitCode=6
Jul 17 14:21:14.712789 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:14.712433    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622" event={"ID":"a91270dc-3224-41a1-b837-7dbaa0124da4","Type":"ContainerDied","Data":"4aa725a475561f340fc38c90ab1edfcf76910f9d795c87c4c329c705ff44ed4a"}
Jul 17 14:21:14.712789 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:14.712489    2561 scope.go:117] "RemoveContainer" containerID="06fa1b0768becac2716a7f49a9aeed4f4bba5ae1476307f09049fc3ac6c5d7ab"
Jul 17 14:21:14.712789 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:14.712660    2561 scope.go:117] "RemoveContainer" containerID="4aa725a475561f340fc38c90ab1edfcf76910f9d795c87c4c329c705ff44ed4a"
Jul 17 14:21:14.712902 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:21:14.712866    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-xr622_odh-ai-gateway-infra(a91270dc-3224-41a1-b837-7dbaa0124da4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4"
Jul 17 14:21:15.718733 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:15.718708    2561 scope.go:117] "RemoveContainer" containerID="4aa725a475561f340fc38c90ab1edfcf76910f9d795c87c4c329c705ff44ed4a"
Jul 17 14:21:15.719138 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:21:15.718899    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-xr622_odh-ai-gateway-infra(a91270dc-3224-41a1-b837-7dbaa0124da4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4"
Jul 17 14:21:17.765735 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:17.765707    2561 scope.go:117] "RemoveContainer" containerID="56499abeb2f1ca66866173e4e90bd5eec82103143e18d20283a7cec6a712f104"
Jul 17 14:21:18.735331 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:18.735301    2561 generic.go:356] "Generic (PLEG): container finished" podID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerID="c844df2315a9eba4dc4343feaf1a2e7ab7ca82e8eb898e405f154402d0194d88" exitCode=7
Jul 17 14:21:18.735504 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:18.735368    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd" event={"ID":"36898e2b-fd0a-45b3-a296-6ba170589e41","Type":"ContainerDied","Data":"c844df2315a9eba4dc4343feaf1a2e7ab7ca82e8eb898e405f154402d0194d88"}
Jul 17 14:21:18.735504 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:18.735426    2561 scope.go:117] "RemoveContainer" containerID="56499abeb2f1ca66866173e4e90bd5eec82103143e18d20283a7cec6a712f104"
Jul 17 14:21:18.735727 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:18.735708    2561 scope.go:117] "RemoveContainer" containerID="c844df2315a9eba4dc4343feaf1a2e7ab7ca82e8eb898e405f154402d0194d88"
Jul 17 14:21:18.735962 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:21:18.735932    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-7vrbd_odh-ai-gateway-infra(36898e2b-fd0a-45b3-a296-6ba170589e41)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41"
Jul 17 14:21:19.762254 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:19.762222    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"]
Jul 17 14:21:19.910290 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:19.910261    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"
Jul 17 14:21:19.998310 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:19.998284    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36898e2b-fd0a-45b3-a296-6ba170589e41-maas-api-service-ca\") pod \"36898e2b-fd0a-45b3-a296-6ba170589e41\" (UID: \"36898e2b-fd0a-45b3-a296-6ba170589e41\") "
Jul 17 14:21:19.998607 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:19.998587    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36898e2b-fd0a-45b3-a296-6ba170589e41-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "36898e2b-fd0a-45b3-a296-6ba170589e41" (UID: "36898e2b-fd0a-45b3-a296-6ba170589e41"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:21:20.099421 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:20.099361    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36898e2b-fd0a-45b3-a296-6ba170589e41-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:21:20.748356 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:20.748327    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"
Jul 17 14:21:20.748522 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:20.748330    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd" event={"ID":"36898e2b-fd0a-45b3-a296-6ba170589e41","Type":"ContainerDied","Data":"f52259921b3145838f8f8281cf3b8e8395f579eb4585984929426c46a1cfbbb4"}
Jul 17 14:21:20.748522 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:20.748461    2561 scope.go:117] "RemoveContainer" containerID="c844df2315a9eba4dc4343feaf1a2e7ab7ca82e8eb898e405f154402d0194d88"
Jul 17 14:21:20.774315 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:20.774293    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"]
Jul 17 14:21:20.779893 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:20.779865    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-7vrbd"]
Jul 17 14:21:22.771843 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:22.771813    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41" path="/var/lib/kubelet/pods/36898e2b-fd0a-45b3-a296-6ba170589e41/volumes"
Jul 17 14:21:30.766141 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:30.766110    2561 scope.go:117] "RemoveContainer" containerID="4aa725a475561f340fc38c90ab1edfcf76910f9d795c87c4c329c705ff44ed4a"
Jul 17 14:21:31.808831 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:31.808801    2561 generic.go:356] "Generic (PLEG): container finished" podID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerID="bf7ec0ac6b9af1b98ed631bb048c608d3a2f295ce318834187f0840510bbccda" exitCode=6
Jul 17 14:21:31.809304 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:31.808843    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622" event={"ID":"a91270dc-3224-41a1-b837-7dbaa0124da4","Type":"ContainerDied","Data":"bf7ec0ac6b9af1b98ed631bb048c608d3a2f295ce318834187f0840510bbccda"}
Jul 17 14:21:31.809304 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:31.808896    2561 scope.go:117] "RemoveContainer" containerID="4aa725a475561f340fc38c90ab1edfcf76910f9d795c87c4c329c705ff44ed4a"
Jul 17 14:21:31.809304 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:31.809162    2561 scope.go:117] "RemoveContainer" containerID="bf7ec0ac6b9af1b98ed631bb048c608d3a2f295ce318834187f0840510bbccda"
Jul 17 14:21:31.809428 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:21:31.809376    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-xr622_odh-ai-gateway-infra(a91270dc-3224-41a1-b837-7dbaa0124da4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4"
Jul 17 14:21:32.834884 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:32.834850    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"]
Jul 17 14:21:32.989509 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:32.989488    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"
Jul 17 14:21:33.115394 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:33.115320    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a91270dc-3224-41a1-b837-7dbaa0124da4-maas-api-service-ca\") pod \"a91270dc-3224-41a1-b837-7dbaa0124da4\" (UID: \"a91270dc-3224-41a1-b837-7dbaa0124da4\") "
Jul 17 14:21:33.115710 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:33.115688    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a91270dc-3224-41a1-b837-7dbaa0124da4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a91270dc-3224-41a1-b837-7dbaa0124da4" (UID: "a91270dc-3224-41a1-b837-7dbaa0124da4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:21:33.216488 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:33.216459    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a91270dc-3224-41a1-b837-7dbaa0124da4-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:21:33.822146 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:33.822117    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"
Jul 17 14:21:33.822315 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:33.822152    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622" event={"ID":"a91270dc-3224-41a1-b837-7dbaa0124da4","Type":"ContainerDied","Data":"c4ba4ed026e4d44cd25994df2d9c72fc9fb4d334457c36ec289ba914501bdeb7"}
Jul 17 14:21:33.822315 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:33.822216    2561 scope.go:117] "RemoveContainer" containerID="bf7ec0ac6b9af1b98ed631bb048c608d3a2f295ce318834187f0840510bbccda"
Jul 17 14:21:33.848150 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:33.848120    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"]
Jul 17 14:21:33.855307 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:33.855280    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-xr622"]
Jul 17 14:21:34.773084 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:34.773049    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4" path="/var/lib/kubelet/pods/a91270dc-3224-41a1-b837-7dbaa0124da4/volumes"
Jul 17 14:21:52.797297 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797261    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"]
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797670    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="286d316b-41ab-4275-a713-bc397bdd01b8" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797680    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="286d316b-41ab-4275-a713-bc397bdd01b8" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797695    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797700    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797711    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797717    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797725    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="286d316b-41ab-4275-a713-bc397bdd01b8" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797730    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="286d316b-41ab-4275-a713-bc397bdd01b8" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797734    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797740    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797751    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797756    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797761    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerName="revoke-keys"
Jul 17 14:21:52.797779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797766    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerName="revoke-keys"
Jul 17 14:21:52.798251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797832    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerName="revoke-keys"
Jul 17 14:21:52.798251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797840    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="286d316b-41ab-4275-a713-bc397bdd01b8" containerName="revoke-keys"
Jul 17 14:21:52.798251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797847    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerName="revoke-keys"
Jul 17 14:21:52.798251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797852    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="286d316b-41ab-4275-a713-bc397bdd01b8" containerName="revoke-keys"
Jul 17 14:21:52.798251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797859    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerName="revoke-keys"
Jul 17 14:21:52.798251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797863    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="286d316b-41ab-4275-a713-bc397bdd01b8" containerName="revoke-keys"
Jul 17 14:21:52.798251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.797869    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerName="revoke-keys"
Jul 17 14:21:52.800968 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.800951    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"
Jul 17 14:21:52.804051 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.804027    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:21:52.804339 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.804323    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:21:52.813520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.813498    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"]
Jul 17 14:21:52.903896 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:52.903863    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/709522a1-52fe-45c6-bc2c-ee31cd51080f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct\" (UID: \"709522a1-52fe-45c6-bc2c-ee31cd51080f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"
Jul 17 14:21:53.005380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:53.005330    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/709522a1-52fe-45c6-bc2c-ee31cd51080f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct\" (UID: \"709522a1-52fe-45c6-bc2c-ee31cd51080f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"
Jul 17 14:21:53.005973 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:53.005950    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/709522a1-52fe-45c6-bc2c-ee31cd51080f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct\" (UID: \"709522a1-52fe-45c6-bc2c-ee31cd51080f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"
Jul 17 14:21:53.111193 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:53.111106    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"
Jul 17 14:21:53.253689 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:53.253659    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"]
Jul 17 14:21:53.257298 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:21:53.257271    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709522a1_52fe_45c6_bc2c_ee31cd51080f.slice/crio-fd0799d5eb486a98b9152b58f5fa5b78ac57b92a148f769041108bcf47204a17 WatchSource:0}: Error finding container fd0799d5eb486a98b9152b58f5fa5b78ac57b92a148f769041108bcf47204a17: Status 404 returned error can't find the container with id fd0799d5eb486a98b9152b58f5fa5b78ac57b92a148f769041108bcf47204a17
Jul 17 14:21:53.945407 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:53.945374    2561 generic.go:356] "Generic (PLEG): container finished" podID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerID="74f409cd84ce2d888c0f1785e404025ad7210b618b6a74f6b14ecfd21e8217da" exitCode=6
Jul 17 14:21:53.945763 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:53.945431    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct" event={"ID":"709522a1-52fe-45c6-bc2c-ee31cd51080f","Type":"ContainerDied","Data":"74f409cd84ce2d888c0f1785e404025ad7210b618b6a74f6b14ecfd21e8217da"}
Jul 17 14:21:53.945763 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:53.945484    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct" event={"ID":"709522a1-52fe-45c6-bc2c-ee31cd51080f","Type":"ContainerStarted","Data":"fd0799d5eb486a98b9152b58f5fa5b78ac57b92a148f769041108bcf47204a17"}
Jul 17 14:21:53.945763 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:53.945659    2561 scope.go:117] "RemoveContainer" containerID="74f409cd84ce2d888c0f1785e404025ad7210b618b6a74f6b14ecfd21e8217da"
Jul 17 14:21:54.952698 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:54.952660    2561 generic.go:356] "Generic (PLEG): container finished" podID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerID="dd7fac8746842c5e5847bb61fb5cb36678e345bc78b5ffb41369ac1155765030" exitCode=6
Jul 17 14:21:54.953178 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:54.952753    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct" event={"ID":"709522a1-52fe-45c6-bc2c-ee31cd51080f","Type":"ContainerDied","Data":"dd7fac8746842c5e5847bb61fb5cb36678e345bc78b5ffb41369ac1155765030"}
Jul 17 14:21:54.953178 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:54.952817    2561 scope.go:117] "RemoveContainer" containerID="74f409cd84ce2d888c0f1785e404025ad7210b618b6a74f6b14ecfd21e8217da"
Jul 17 14:21:54.953178 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:54.952963    2561 scope.go:117] "RemoveContainer" containerID="dd7fac8746842c5e5847bb61fb5cb36678e345bc78b5ffb41369ac1155765030"
Jul 17 14:21:54.953299 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:21:54.953285    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct_odh-ai-gateway-infra(709522a1-52fe-45c6-bc2c-ee31cd51080f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f"
Jul 17 14:21:55.959753 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:21:55.959723    2561 scope.go:117] "RemoveContainer" containerID="dd7fac8746842c5e5847bb61fb5cb36678e345bc78b5ffb41369ac1155765030"
Jul 17 14:21:55.960172 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:21:55.959968    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct_odh-ai-gateway-infra(709522a1-52fe-45c6-bc2c-ee31cd51080f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f"
Jul 17 14:22:09.765881 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:09.765852    2561 scope.go:117] "RemoveContainer" containerID="dd7fac8746842c5e5847bb61fb5cb36678e345bc78b5ffb41369ac1155765030"
Jul 17 14:22:10.047254 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:10.047182    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct" event={"ID":"709522a1-52fe-45c6-bc2c-ee31cd51080f","Type":"ContainerStarted","Data":"991849c9ba65ef192ff10534a29eaa7905ecae8cd78cc50706565c1eee99771d"}
Jul 17 14:22:10.062679 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:10.062560    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct" podStartSLOduration=18.062540049 podStartE2EDuration="18.062540049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:21:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:22:10.062515632 +0000 UTC m=+2215.768962467" watchObservedRunningTime="2026-07-17 14:22:10.062540049 +0000 UTC m=+2215.768986884"
Jul 17 14:22:11.053976 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.053940    2561 generic.go:356] "Generic (PLEG): container finished" podID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerID="991849c9ba65ef192ff10534a29eaa7905ecae8cd78cc50706565c1eee99771d" exitCode=6
Jul 17 14:22:11.054413 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.054028    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct" event={"ID":"709522a1-52fe-45c6-bc2c-ee31cd51080f","Type":"ContainerDied","Data":"991849c9ba65ef192ff10534a29eaa7905ecae8cd78cc50706565c1eee99771d"}
Jul 17 14:22:11.054413 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.054097    2561 scope.go:117] "RemoveContainer" containerID="dd7fac8746842c5e5847bb61fb5cb36678e345bc78b5ffb41369ac1155765030"
Jul 17 14:22:11.054413 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.054303    2561 scope.go:117] "RemoveContainer" containerID="991849c9ba65ef192ff10534a29eaa7905ecae8cd78cc50706565c1eee99771d"
Jul 17 14:22:11.054575 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:22:11.054552    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct_odh-ai-gateway-infra(709522a1-52fe-45c6-bc2c-ee31cd51080f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f"
Jul 17 14:22:11.076029 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.075988    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"]
Jul 17 14:22:11.931708 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.931675    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"]
Jul 17 14:22:11.932162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.932148    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="286d316b-41ab-4275-a713-bc397bdd01b8" containerName="revoke-keys"
Jul 17 14:22:11.932162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.932163    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="286d316b-41ab-4275-a713-bc397bdd01b8" containerName="revoke-keys"
Jul 17 14:22:11.932260 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.932175    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerName="revoke-keys"
Jul 17 14:22:11.932260 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.932180    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerName="revoke-keys"
Jul 17 14:22:11.932260 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.932243    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a91270dc-3224-41a1-b837-7dbaa0124da4" containerName="revoke-keys"
Jul 17 14:22:11.932260 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.932254    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="36898e2b-fd0a-45b3-a296-6ba170589e41" containerName="revoke-keys"
Jul 17 14:22:11.936420 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.936404    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"
Jul 17 14:22:11.961000 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:11.960972    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"]
Jul 17 14:22:12.091422 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:12.091377    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56a305b7-56d8-4077-80d7-07ab3c3fe818-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq\" (UID: \"56a305b7-56d8-4077-80d7-07ab3c3fe818\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"
Jul 17 14:22:12.192264 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:12.192236    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56a305b7-56d8-4077-80d7-07ab3c3fe818-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq\" (UID: \"56a305b7-56d8-4077-80d7-07ab3c3fe818\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"
Jul 17 14:22:12.192819 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:12.192794    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56a305b7-56d8-4077-80d7-07ab3c3fe818-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq\" (UID: \"56a305b7-56d8-4077-80d7-07ab3c3fe818\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"
Jul 17 14:22:12.199502 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:12.199484    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"
Jul 17 14:22:12.246625 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:12.246590    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"
Jul 17 14:22:12.293489 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:12.293464    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/709522a1-52fe-45c6-bc2c-ee31cd51080f-maas-api-service-ca\") pod \"709522a1-52fe-45c6-bc2c-ee31cd51080f\" (UID: \"709522a1-52fe-45c6-bc2c-ee31cd51080f\") "
Jul 17 14:22:12.293829 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:12.293806    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/709522a1-52fe-45c6-bc2c-ee31cd51080f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "709522a1-52fe-45c6-bc2c-ee31cd51080f" (UID: "709522a1-52fe-45c6-bc2c-ee31cd51080f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:22:12.385628 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:12.385419    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"]
Jul 17 14:22:12.387931 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:22:12.387900    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a305b7_56d8_4077_80d7_07ab3c3fe818.slice/crio-5c089a5d87d90013a667ba82a35200e409eda9216287254625b2a12f2fb8e695 WatchSource:0}: Error finding container 5c089a5d87d90013a667ba82a35200e409eda9216287254625b2a12f2fb8e695: Status 404 returned error can't find the container with id 5c089a5d87d90013a667ba82a35200e409eda9216287254625b2a12f2fb8e695
Jul 17 14:22:12.394836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:12.394813    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/709522a1-52fe-45c6-bc2c-ee31cd51080f-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:22:13.067688 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:13.067664    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"
Jul 17 14:22:13.067855 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:13.067693    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct" event={"ID":"709522a1-52fe-45c6-bc2c-ee31cd51080f","Type":"ContainerDied","Data":"fd0799d5eb486a98b9152b58f5fa5b78ac57b92a148f769041108bcf47204a17"}
Jul 17 14:22:13.067855 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:13.067756    2561 scope.go:117] "RemoveContainer" containerID="991849c9ba65ef192ff10534a29eaa7905ecae8cd78cc50706565c1eee99771d"
Jul 17 14:22:13.069454 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:13.069429    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" event={"ID":"56a305b7-56d8-4077-80d7-07ab3c3fe818","Type":"ContainerStarted","Data":"f18f584d6008d6580f3d0c58c9fc2240513a69b48aa7b2ac804a57e621d72bd8"}
Jul 17 14:22:13.069559 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:13.069491    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" event={"ID":"56a305b7-56d8-4077-80d7-07ab3c3fe818","Type":"ContainerStarted","Data":"5c089a5d87d90013a667ba82a35200e409eda9216287254625b2a12f2fb8e695"}
Jul 17 14:22:13.086102 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:13.086076    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"]
Jul 17 14:22:13.094428 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:13.094409    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-pd2ct"]
Jul 17 14:22:13.108718 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:13.108678    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" podStartSLOduration=2.1086651 podStartE2EDuration="2.1086651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:22:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:22:13.107092938 +0000 UTC m=+2218.813539771" watchObservedRunningTime="2026-07-17 14:22:13.1086651 +0000 UTC m=+2218.815111934"
Jul 17 14:22:14.076758 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:14.076717    2561 generic.go:356] "Generic (PLEG): container finished" podID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerID="f18f584d6008d6580f3d0c58c9fc2240513a69b48aa7b2ac804a57e621d72bd8" exitCode=6
Jul 17 14:22:14.076923 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:14.076805    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" event={"ID":"56a305b7-56d8-4077-80d7-07ab3c3fe818","Type":"ContainerDied","Data":"f18f584d6008d6580f3d0c58c9fc2240513a69b48aa7b2ac804a57e621d72bd8"}
Jul 17 14:22:14.077036 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:14.077020    2561 scope.go:117] "RemoveContainer" containerID="f18f584d6008d6580f3d0c58c9fc2240513a69b48aa7b2ac804a57e621d72bd8"
Jul 17 14:22:14.772660 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:14.772632    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f" path="/var/lib/kubelet/pods/709522a1-52fe-45c6-bc2c-ee31cd51080f/volumes"
Jul 17 14:22:15.083516 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:15.083488    2561 generic.go:356] "Generic (PLEG): container finished" podID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerID="e9a6b0414f7799b27c48d5c9d9ded8202e17db26eb5e3df1702bdd293434ab4e" exitCode=6
Jul 17 14:22:15.083635 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:15.083568    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" event={"ID":"56a305b7-56d8-4077-80d7-07ab3c3fe818","Type":"ContainerDied","Data":"e9a6b0414f7799b27c48d5c9d9ded8202e17db26eb5e3df1702bdd293434ab4e"}
Jul 17 14:22:15.083635 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:15.083623    2561 scope.go:117] "RemoveContainer" containerID="f18f584d6008d6580f3d0c58c9fc2240513a69b48aa7b2ac804a57e621d72bd8"
Jul 17 14:22:15.083794 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:15.083777    2561 scope.go:117] "RemoveContainer" containerID="e9a6b0414f7799b27c48d5c9d9ded8202e17db26eb5e3df1702bdd293434ab4e"
Jul 17 14:22:15.084030 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:22:15.083987    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq_odh-ai-gateway-infra(56a305b7-56d8-4077-80d7-07ab3c3fe818)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818"
Jul 17 14:22:16.090655 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:16.090629    2561 scope.go:117] "RemoveContainer" containerID="e9a6b0414f7799b27c48d5c9d9ded8202e17db26eb5e3df1702bdd293434ab4e"
Jul 17 14:22:16.091057 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:22:16.090812    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq_odh-ai-gateway-infra(56a305b7-56d8-4077-80d7-07ab3c3fe818)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818"
Jul 17 14:22:27.765742 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:27.765710    2561 scope.go:117] "RemoveContainer" containerID="e9a6b0414f7799b27c48d5c9d9ded8202e17db26eb5e3df1702bdd293434ab4e"
Jul 17 14:22:28.153401 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:28.153326    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" event={"ID":"56a305b7-56d8-4077-80d7-07ab3c3fe818","Type":"ContainerStarted","Data":"dac17e222cc638a3e56f5d29cd0b3100504a293d18b0cb2ce114f521690a64e7"}
Jul 17 14:22:29.160985 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:29.160948    2561 generic.go:356] "Generic (PLEG): container finished" podID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerID="dac17e222cc638a3e56f5d29cd0b3100504a293d18b0cb2ce114f521690a64e7" exitCode=6
Jul 17 14:22:29.161439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:29.161045    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" event={"ID":"56a305b7-56d8-4077-80d7-07ab3c3fe818","Type":"ContainerDied","Data":"dac17e222cc638a3e56f5d29cd0b3100504a293d18b0cb2ce114f521690a64e7"}
Jul 17 14:22:29.161439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:29.161111    2561 scope.go:117] "RemoveContainer" containerID="e9a6b0414f7799b27c48d5c9d9ded8202e17db26eb5e3df1702bdd293434ab4e"
Jul 17 14:22:29.161439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:29.161394    2561 scope.go:117] "RemoveContainer" containerID="dac17e222cc638a3e56f5d29cd0b3100504a293d18b0cb2ce114f521690a64e7"
Jul 17 14:22:29.161621 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:22:29.161583    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq_odh-ai-gateway-infra(56a305b7-56d8-4077-80d7-07ab3c3fe818)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818"
Jul 17 14:22:29.185150 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:29.185122    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"]
Jul 17 14:22:30.322451 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:30.322427    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"
Jul 17 14:22:30.453101 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:30.453030    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56a305b7-56d8-4077-80d7-07ab3c3fe818-maas-api-service-ca\") pod \"56a305b7-56d8-4077-80d7-07ab3c3fe818\" (UID: \"56a305b7-56d8-4077-80d7-07ab3c3fe818\") "
Jul 17 14:22:30.453395 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:30.453372    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56a305b7-56d8-4077-80d7-07ab3c3fe818-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "56a305b7-56d8-4077-80d7-07ab3c3fe818" (UID: "56a305b7-56d8-4077-80d7-07ab3c3fe818"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:22:30.553932 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:30.553907    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56a305b7-56d8-4077-80d7-07ab3c3fe818-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:22:31.174286 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:31.174262    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"
Jul 17 14:22:31.174286 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:31.174253    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq" event={"ID":"56a305b7-56d8-4077-80d7-07ab3c3fe818","Type":"ContainerDied","Data":"5c089a5d87d90013a667ba82a35200e409eda9216287254625b2a12f2fb8e695"}
Jul 17 14:22:31.174493 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:31.174340    2561 scope.go:117] "RemoveContainer" containerID="dac17e222cc638a3e56f5d29cd0b3100504a293d18b0cb2ce114f521690a64e7"
Jul 17 14:22:31.207900 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:31.207875    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"]
Jul 17 14:22:31.215606 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:31.215582    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-lq6gq"]
Jul 17 14:22:32.772146 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:22:32.772114    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818" path="/var/lib/kubelet/pods/56a305b7-56d8-4077-80d7-07ab3c3fe818/volumes"
Jul 17 14:23:01.954608 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954530    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"]
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954925    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954936    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954945    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954950    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954964    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954969    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954976    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954980    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954988    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.954993    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.955080    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.955089    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerName="revoke-keys"
Jul 17 14:23:01.955090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.955095    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerName="revoke-keys"
Jul 17 14:23:01.955537 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.955102    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerName="revoke-keys"
Jul 17 14:23:01.955537 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.955108    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerName="revoke-keys"
Jul 17 14:23:01.957757 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.957741    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"
Jul 17 14:23:01.961691 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.961666    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:23:01.961828 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.961704    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:23:01.967421 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:01.967398    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"]
Jul 17 14:23:02.023090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:02.023060    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/380cdf79-906f-4457-adab-c18de28cac43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn\" (UID: \"380cdf79-906f-4457-adab-c18de28cac43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"
Jul 17 14:23:02.123809 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:02.123780    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/380cdf79-906f-4457-adab-c18de28cac43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn\" (UID: \"380cdf79-906f-4457-adab-c18de28cac43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"
Jul 17 14:23:02.124386 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:02.124368    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/380cdf79-906f-4457-adab-c18de28cac43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn\" (UID: \"380cdf79-906f-4457-adab-c18de28cac43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"
Jul 17 14:23:02.269510 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:02.269449    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"
Jul 17 14:23:02.617182 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:02.617149    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"]
Jul 17 14:23:02.623735 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:23:02.623703    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod380cdf79_906f_4457_adab_c18de28cac43.slice/crio-5e1ff0fb5a61b31753600da4aa3ebebca8da68700977c7549f30c6478caff654 WatchSource:0}: Error finding container 5e1ff0fb5a61b31753600da4aa3ebebca8da68700977c7549f30c6478caff654: Status 404 returned error can't find the container with id 5e1ff0fb5a61b31753600da4aa3ebebca8da68700977c7549f30c6478caff654
Jul 17 14:23:03.346441 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:03.346411    2561 generic.go:356] "Generic (PLEG): container finished" podID="380cdf79-906f-4457-adab-c18de28cac43" containerID="f53074457c0bb927468718e111735e5fdefebc65ff076254d47bf79e30d0ab17" exitCode=6
Jul 17 14:23:03.346781 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:03.346501    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn" event={"ID":"380cdf79-906f-4457-adab-c18de28cac43","Type":"ContainerDied","Data":"f53074457c0bb927468718e111735e5fdefebc65ff076254d47bf79e30d0ab17"}
Jul 17 14:23:03.346781 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:03.346547    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn" event={"ID":"380cdf79-906f-4457-adab-c18de28cac43","Type":"ContainerStarted","Data":"5e1ff0fb5a61b31753600da4aa3ebebca8da68700977c7549f30c6478caff654"}
Jul 17 14:23:03.346781 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:03.346629    2561 scope.go:117] "RemoveContainer" containerID="f53074457c0bb927468718e111735e5fdefebc65ff076254d47bf79e30d0ab17"
Jul 17 14:23:04.353884 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:04.353804    2561 generic.go:356] "Generic (PLEG): container finished" podID="380cdf79-906f-4457-adab-c18de28cac43" containerID="8ecf9599138f7999b3731eb62bfd8caae0c3fee067e64e8f6e201e217ae100fe" exitCode=6
Jul 17 14:23:04.354322 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:04.353882    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn" event={"ID":"380cdf79-906f-4457-adab-c18de28cac43","Type":"ContainerDied","Data":"8ecf9599138f7999b3731eb62bfd8caae0c3fee067e64e8f6e201e217ae100fe"}
Jul 17 14:23:04.354322 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:04.353937    2561 scope.go:117] "RemoveContainer" containerID="f53074457c0bb927468718e111735e5fdefebc65ff076254d47bf79e30d0ab17"
Jul 17 14:23:04.354322 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:04.354100    2561 scope.go:117] "RemoveContainer" containerID="8ecf9599138f7999b3731eb62bfd8caae0c3fee067e64e8f6e201e217ae100fe"
Jul 17 14:23:04.354322 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:23:04.354310    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn_odh-ai-gateway-infra(380cdf79-906f-4457-adab-c18de28cac43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn" podUID="380cdf79-906f-4457-adab-c18de28cac43"
Jul 17 14:23:05.360650 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:05.360620    2561 scope.go:117] "RemoveContainer" containerID="8ecf9599138f7999b3731eb62bfd8caae0c3fee067e64e8f6e201e217ae100fe"
Jul 17 14:23:05.361064 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:23:05.360832    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn_odh-ai-gateway-infra(380cdf79-906f-4457-adab-c18de28cac43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn" podUID="380cdf79-906f-4457-adab-c18de28cac43"
Jul 17 14:23:17.766488 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:17.766461    2561 scope.go:117] "RemoveContainer" containerID="8ecf9599138f7999b3731eb62bfd8caae0c3fee067e64e8f6e201e217ae100fe"
Jul 17 14:23:18.433061 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:18.433028    2561 generic.go:356] "Generic (PLEG): container finished" podID="380cdf79-906f-4457-adab-c18de28cac43" containerID="f2ccf9b8d020afa310040c02bae83ead7bcae967825e9ca4227db80a1b6ce6d0" exitCode=6
Jul 17 14:23:18.433190 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:18.433066    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn" event={"ID":"380cdf79-906f-4457-adab-c18de28cac43","Type":"ContainerDied","Data":"f2ccf9b8d020afa310040c02bae83ead7bcae967825e9ca4227db80a1b6ce6d0"}
Jul 17 14:23:18.433190 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:18.433124    2561 scope.go:117] "RemoveContainer" containerID="8ecf9599138f7999b3731eb62bfd8caae0c3fee067e64e8f6e201e217ae100fe"
Jul 17 14:23:18.433433 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:18.433414    2561 scope.go:117] "RemoveContainer" containerID="f2ccf9b8d020afa310040c02bae83ead7bcae967825e9ca4227db80a1b6ce6d0"
Jul 17 14:23:18.433644 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:23:18.433621    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn_odh-ai-gateway-infra(380cdf79-906f-4457-adab-c18de28cac43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn" podUID="380cdf79-906f-4457-adab-c18de28cac43"
Jul 17 14:23:19.461075 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:19.461040    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"]
Jul 17 14:23:19.597132 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:19.597105    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"
Jul 17 14:23:19.691782 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:19.691752    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/380cdf79-906f-4457-adab-c18de28cac43-maas-api-service-ca\") pod \"380cdf79-906f-4457-adab-c18de28cac43\" (UID: \"380cdf79-906f-4457-adab-c18de28cac43\") "
Jul 17 14:23:19.692099 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:19.692078    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380cdf79-906f-4457-adab-c18de28cac43-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "380cdf79-906f-4457-adab-c18de28cac43" (UID: "380cdf79-906f-4457-adab-c18de28cac43"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:23:19.792859 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:19.792801    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/380cdf79-906f-4457-adab-c18de28cac43-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:23:20.447183 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:20.447154    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"
Jul 17 14:23:20.447360 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:20.447151    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn" event={"ID":"380cdf79-906f-4457-adab-c18de28cac43","Type":"ContainerDied","Data":"5e1ff0fb5a61b31753600da4aa3ebebca8da68700977c7549f30c6478caff654"}
Jul 17 14:23:20.447360 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:20.447296    2561 scope.go:117] "RemoveContainer" containerID="f2ccf9b8d020afa310040c02bae83ead7bcae967825e9ca4227db80a1b6ce6d0"
Jul 17 14:23:20.469456 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:20.469433    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"]
Jul 17 14:23:20.477127 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:20.477106    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-z4rtn"]
Jul 17 14:23:20.772669 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:20.772590    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="380cdf79-906f-4457-adab-c18de28cac43" path="/var/lib/kubelet/pods/380cdf79-906f-4457-adab-c18de28cac43/volumes"
Jul 17 14:23:41.264315 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264277    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"]
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264689    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="380cdf79-906f-4457-adab-c18de28cac43" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264701    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="380cdf79-906f-4457-adab-c18de28cac43" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264721    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="380cdf79-906f-4457-adab-c18de28cac43" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264726    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="380cdf79-906f-4457-adab-c18de28cac43" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264735    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264741    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="709522a1-52fe-45c6-bc2c-ee31cd51080f" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264754    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="380cdf79-906f-4457-adab-c18de28cac43" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264758    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="380cdf79-906f-4457-adab-c18de28cac43" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264820    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="380cdf79-906f-4457-adab-c18de28cac43" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264830    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a305b7-56d8-4077-80d7-07ab3c3fe818" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264836    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="380cdf79-906f-4457-adab-c18de28cac43" containerName="revoke-keys"
Jul 17 14:23:41.264836 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.264842    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="380cdf79-906f-4457-adab-c18de28cac43" containerName="revoke-keys"
Jul 17 14:23:41.269287 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.269265    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"
Jul 17 14:23:41.272417 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.272396    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:23:41.272897 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.272878    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:23:41.282739 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.282713    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"]
Jul 17 14:23:41.394677 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.394632    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0db679d-588b-479e-810d-ecaac987eb82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-ql8hd\" (UID: \"a0db679d-588b-479e-810d-ecaac987eb82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"
Jul 17 14:23:41.495940 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.495895    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0db679d-588b-479e-810d-ecaac987eb82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-ql8hd\" (UID: \"a0db679d-588b-479e-810d-ecaac987eb82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"
Jul 17 14:23:41.496572 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.496551    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0db679d-588b-479e-810d-ecaac987eb82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-ql8hd\" (UID: \"a0db679d-588b-479e-810d-ecaac987eb82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"
Jul 17 14:23:41.580559 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.580477    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"
Jul 17 14:23:41.724429 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:41.724390    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"]
Jul 17 14:23:41.725537 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:23:41.725506    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0db679d_588b_479e_810d_ecaac987eb82.slice/crio-b600682324e63f2c8d3a093c57f1f9c7317b895e48b2ae662721403de38b7d12 WatchSource:0}: Error finding container b600682324e63f2c8d3a093c57f1f9c7317b895e48b2ae662721403de38b7d12: Status 404 returned error can't find the container with id b600682324e63f2c8d3a093c57f1f9c7317b895e48b2ae662721403de38b7d12
Jul 17 14:23:42.578326 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:42.578227    2561 generic.go:356] "Generic (PLEG): container finished" podID="a0db679d-588b-479e-810d-ecaac987eb82" containerID="c2a987a224c719b3d6394a071c12271db632a60d202e6a507db52e1619c83fca" exitCode=7
Jul 17 14:23:42.578326 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:42.578276    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd" event={"ID":"a0db679d-588b-479e-810d-ecaac987eb82","Type":"ContainerDied","Data":"c2a987a224c719b3d6394a071c12271db632a60d202e6a507db52e1619c83fca"}
Jul 17 14:23:42.578850 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:42.578375    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd" event={"ID":"a0db679d-588b-479e-810d-ecaac987eb82","Type":"ContainerStarted","Data":"b600682324e63f2c8d3a093c57f1f9c7317b895e48b2ae662721403de38b7d12"}
Jul 17 14:23:42.578850 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:42.578613    2561 scope.go:117] "RemoveContainer" containerID="c2a987a224c719b3d6394a071c12271db632a60d202e6a507db52e1619c83fca"
Jul 17 14:23:43.585704 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:43.585619    2561 generic.go:356] "Generic (PLEG): container finished" podID="a0db679d-588b-479e-810d-ecaac987eb82" containerID="34ffeb26506ffdd073ffc266f3e82f19758c88f88477f6b961c6ca4317e1fa39" exitCode=7
Jul 17 14:23:43.585704 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:43.585677    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd" event={"ID":"a0db679d-588b-479e-810d-ecaac987eb82","Type":"ContainerDied","Data":"34ffeb26506ffdd073ffc266f3e82f19758c88f88477f6b961c6ca4317e1fa39"}
Jul 17 14:23:43.586207 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:43.585724    2561 scope.go:117] "RemoveContainer" containerID="c2a987a224c719b3d6394a071c12271db632a60d202e6a507db52e1619c83fca"
Jul 17 14:23:43.586207 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:43.585879    2561 scope.go:117] "RemoveContainer" containerID="34ffeb26506ffdd073ffc266f3e82f19758c88f88477f6b961c6ca4317e1fa39"
Jul 17 14:23:43.586288 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:23:43.586205    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-ql8hd_odh-ai-gateway-infra(a0db679d-588b-479e-810d-ecaac987eb82)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd" podUID="a0db679d-588b-479e-810d-ecaac987eb82"
Jul 17 14:23:44.593203 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:44.593167    2561 scope.go:117] "RemoveContainer" containerID="34ffeb26506ffdd073ffc266f3e82f19758c88f88477f6b961c6ca4317e1fa39"
Jul 17 14:23:44.593583 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:23:44.593393    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-ql8hd_odh-ai-gateway-infra(a0db679d-588b-479e-810d-ecaac987eb82)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd" podUID="a0db679d-588b-479e-810d-ecaac987eb82"
Jul 17 14:23:52.001430 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:52.001392    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"]
Jul 17 14:23:52.004950 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:52.004930    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"
Jul 17 14:23:52.012961 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:52.012934    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"]
Jul 17 14:23:52.105989 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:52.105951    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9961846-7954-4a71-ad88-4d8ba40a3069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7\" (UID: \"e9961846-7954-4a71-ad88-4d8ba40a3069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"
Jul 17 14:23:52.206918 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:52.206869    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9961846-7954-4a71-ad88-4d8ba40a3069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7\" (UID: \"e9961846-7954-4a71-ad88-4d8ba40a3069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"
Jul 17 14:23:52.207533 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:52.207512    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9961846-7954-4a71-ad88-4d8ba40a3069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7\" (UID: \"e9961846-7954-4a71-ad88-4d8ba40a3069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"
Jul 17 14:23:52.317506 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:52.317420    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"
Jul 17 14:23:52.663539 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:52.663513    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"]
Jul 17 14:23:52.665743 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:23:52.665698    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9961846_7954_4a71_ad88_4d8ba40a3069.slice/crio-ac304936f0d72c1a5a147e6c1fbd37590b58f3a9bcee53cf2ab2cb7e203f4b2d WatchSource:0}: Error finding container ac304936f0d72c1a5a147e6c1fbd37590b58f3a9bcee53cf2ab2cb7e203f4b2d: Status 404 returned error can't find the container with id ac304936f0d72c1a5a147e6c1fbd37590b58f3a9bcee53cf2ab2cb7e203f4b2d
Jul 17 14:23:53.646548 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:53.646450    2561 generic.go:356] "Generic (PLEG): container finished" podID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerID="2d70cfd859c2d7aea35da8eb300f2f64c5d5dd4566787afdbc3e9b26165cb358" exitCode=6
Jul 17 14:23:53.646548 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:53.646514    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7" event={"ID":"e9961846-7954-4a71-ad88-4d8ba40a3069","Type":"ContainerDied","Data":"2d70cfd859c2d7aea35da8eb300f2f64c5d5dd4566787afdbc3e9b26165cb358"}
Jul 17 14:23:53.646973 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:53.646599    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7" event={"ID":"e9961846-7954-4a71-ad88-4d8ba40a3069","Type":"ContainerStarted","Data":"ac304936f0d72c1a5a147e6c1fbd37590b58f3a9bcee53cf2ab2cb7e203f4b2d"}
Jul 17 14:23:53.646973 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:53.646850    2561 scope.go:117] "RemoveContainer" containerID="2d70cfd859c2d7aea35da8eb300f2f64c5d5dd4566787afdbc3e9b26165cb358"
Jul 17 14:23:54.653356 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:54.653281    2561 generic.go:356] "Generic (PLEG): container finished" podID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerID="02537966382b2f096f792e68f3e6b5a31ae1922dee451879c0a0b4eac0bfc09a" exitCode=6
Jul 17 14:23:54.653800 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:54.653368    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7" event={"ID":"e9961846-7954-4a71-ad88-4d8ba40a3069","Type":"ContainerDied","Data":"02537966382b2f096f792e68f3e6b5a31ae1922dee451879c0a0b4eac0bfc09a"}
Jul 17 14:23:54.653800 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:54.653432    2561 scope.go:117] "RemoveContainer" containerID="2d70cfd859c2d7aea35da8eb300f2f64c5d5dd4566787afdbc3e9b26165cb358"
Jul 17 14:23:54.653800 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:54.653643    2561 scope.go:117] "RemoveContainer" containerID="02537966382b2f096f792e68f3e6b5a31ae1922dee451879c0a0b4eac0bfc09a"
Jul 17 14:23:54.653950 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:23:54.653872    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7_odh-ai-gateway-infra(e9961846-7954-4a71-ad88-4d8ba40a3069)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069"
Jul 17 14:23:55.660089 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:55.660062    2561 scope.go:117] "RemoveContainer" containerID="02537966382b2f096f792e68f3e6b5a31ae1922dee451879c0a0b4eac0bfc09a"
Jul 17 14:23:55.660461 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:23:55.660226    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7_odh-ai-gateway-infra(e9961846-7954-4a71-ad88-4d8ba40a3069)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069"
Jul 17 14:23:57.765737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:57.765707    2561 scope.go:117] "RemoveContainer" containerID="34ffeb26506ffdd073ffc266f3e82f19758c88f88477f6b961c6ca4317e1fa39"
Jul 17 14:23:58.676668 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:58.676637    2561 generic.go:356] "Generic (PLEG): container finished" podID="a0db679d-588b-479e-810d-ecaac987eb82" containerID="3595d6f10c1c4f424549eed5a2c6eea940e09a328cf25c4311994c8603695959" exitCode=7
Jul 17 14:23:58.676833 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:58.676712    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd" event={"ID":"a0db679d-588b-479e-810d-ecaac987eb82","Type":"ContainerDied","Data":"3595d6f10c1c4f424549eed5a2c6eea940e09a328cf25c4311994c8603695959"}
Jul 17 14:23:58.676833 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:58.676773    2561 scope.go:117] "RemoveContainer" containerID="34ffeb26506ffdd073ffc266f3e82f19758c88f88477f6b961c6ca4317e1fa39"
Jul 17 14:23:58.677108 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:58.677093    2561 scope.go:117] "RemoveContainer" containerID="3595d6f10c1c4f424549eed5a2c6eea940e09a328cf25c4311994c8603695959"
Jul 17 14:23:58.677360 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:23:58.677342    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-ql8hd_odh-ai-gateway-infra(a0db679d-588b-479e-810d-ecaac987eb82)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd" podUID="a0db679d-588b-479e-810d-ecaac987eb82"
Jul 17 14:23:59.705490 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:59.705456    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"]
Jul 17 14:23:59.850059 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:59.850035    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"
Jul 17 14:23:59.880345 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:59.880319    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0db679d-588b-479e-810d-ecaac987eb82-maas-api-service-ca\") pod \"a0db679d-588b-479e-810d-ecaac987eb82\" (UID: \"a0db679d-588b-479e-810d-ecaac987eb82\") "
Jul 17 14:23:59.880635 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:59.880613    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0db679d-588b-479e-810d-ecaac987eb82-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a0db679d-588b-479e-810d-ecaac987eb82" (UID: "a0db679d-588b-479e-810d-ecaac987eb82"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:23:59.981508 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:23:59.981429    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0db679d-588b-479e-810d-ecaac987eb82-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:24:00.690032 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:00.689983    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"
Jul 17 14:24:00.690032 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:00.690025    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd" event={"ID":"a0db679d-588b-479e-810d-ecaac987eb82","Type":"ContainerDied","Data":"b600682324e63f2c8d3a093c57f1f9c7317b895e48b2ae662721403de38b7d12"}
Jul 17 14:24:00.690329 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:00.690091    2561 scope.go:117] "RemoveContainer" containerID="3595d6f10c1c4f424549eed5a2c6eea940e09a328cf25c4311994c8603695959"
Jul 17 14:24:00.711117 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:00.711087    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"]
Jul 17 14:24:00.716883 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:00.716862    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-ql8hd"]
Jul 17 14:24:00.772574 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:00.772544    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0db679d-588b-479e-810d-ecaac987eb82" path="/var/lib/kubelet/pods/a0db679d-588b-479e-810d-ecaac987eb82/volumes"
Jul 17 14:24:06.772027 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:06.771934    2561 scope.go:117] "RemoveContainer" containerID="02537966382b2f096f792e68f3e6b5a31ae1922dee451879c0a0b4eac0bfc09a"
Jul 17 14:24:07.729633 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:07.729602    2561 generic.go:356] "Generic (PLEG): container finished" podID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerID="e400252092e145bc0b666054e3bebdee73758b485668d67fcc5a4a2b3f0b5814" exitCode=6
Jul 17 14:24:07.731755 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:07.729673    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7" event={"ID":"e9961846-7954-4a71-ad88-4d8ba40a3069","Type":"ContainerDied","Data":"e400252092e145bc0b666054e3bebdee73758b485668d67fcc5a4a2b3f0b5814"}
Jul 17 14:24:07.731755 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:07.729734    2561 scope.go:117] "RemoveContainer" containerID="02537966382b2f096f792e68f3e6b5a31ae1922dee451879c0a0b4eac0bfc09a"
Jul 17 14:24:07.731755 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:07.730043    2561 scope.go:117] "RemoveContainer" containerID="e400252092e145bc0b666054e3bebdee73758b485668d67fcc5a4a2b3f0b5814"
Jul 17 14:24:07.731755 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:24:07.730299    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7_odh-ai-gateway-infra(e9961846-7954-4a71-ad88-4d8ba40a3069)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069"
Jul 17 14:24:08.756080 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:08.756048    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"]
Jul 17 14:24:08.897770 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:08.897749    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"
Jul 17 14:24:08.963242 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:08.963213    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9961846-7954-4a71-ad88-4d8ba40a3069-maas-api-service-ca\") pod \"e9961846-7954-4a71-ad88-4d8ba40a3069\" (UID: \"e9961846-7954-4a71-ad88-4d8ba40a3069\") "
Jul 17 14:24:08.963542 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:08.963520    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9961846-7954-4a71-ad88-4d8ba40a3069-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e9961846-7954-4a71-ad88-4d8ba40a3069" (UID: "e9961846-7954-4a71-ad88-4d8ba40a3069"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:24:09.064815 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:09.064743    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9961846-7954-4a71-ad88-4d8ba40a3069-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:24:09.743560 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:09.743530    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"
Jul 17 14:24:09.743751 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:09.743577    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7" event={"ID":"e9961846-7954-4a71-ad88-4d8ba40a3069","Type":"ContainerDied","Data":"ac304936f0d72c1a5a147e6c1fbd37590b58f3a9bcee53cf2ab2cb7e203f4b2d"}
Jul 17 14:24:09.743751 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:09.743642    2561 scope.go:117] "RemoveContainer" containerID="e400252092e145bc0b666054e3bebdee73758b485668d67fcc5a4a2b3f0b5814"
Jul 17 14:24:09.766985 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:09.766957    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"]
Jul 17 14:24:09.774119 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:09.774100    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-6lrd7"]
Jul 17 14:24:10.771702 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:10.771667    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069" path="/var/lib/kubelet/pods/e9961846-7954-4a71-ad88-4d8ba40a3069/volumes"
Jul 17 14:24:31.331532 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.331497    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"]
Jul 17 14:24:31.331986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.331930    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerName="revoke-keys"
Jul 17 14:24:31.331986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.331941    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerName="revoke-keys"
Jul 17 14:24:31.331986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.331953    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerName="revoke-keys"
Jul 17 14:24:31.331986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.331958    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerName="revoke-keys"
Jul 17 14:24:31.331986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.331967    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0db679d-588b-479e-810d-ecaac987eb82" containerName="revoke-keys"
Jul 17 14:24:31.331986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.331973    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0db679d-588b-479e-810d-ecaac987eb82" containerName="revoke-keys"
Jul 17 14:24:31.331986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.331979    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0db679d-588b-479e-810d-ecaac987eb82" containerName="revoke-keys"
Jul 17 14:24:31.331986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.331985    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0db679d-588b-479e-810d-ecaac987eb82" containerName="revoke-keys"
Jul 17 14:24:31.332265 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.332080    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerName="revoke-keys"
Jul 17 14:24:31.332265 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.332090    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0db679d-588b-479e-810d-ecaac987eb82" containerName="revoke-keys"
Jul 17 14:24:31.332265 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.332097    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerName="revoke-keys"
Jul 17 14:24:31.332265 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.332103    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0db679d-588b-479e-810d-ecaac987eb82" containerName="revoke-keys"
Jul 17 14:24:31.332265 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.332111    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0db679d-588b-479e-810d-ecaac987eb82" containerName="revoke-keys"
Jul 17 14:24:31.335261 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.335243    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"
Jul 17 14:24:31.338569 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.338550    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:24:31.339052 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.339035    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:24:31.361393 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.361365    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"]
Jul 17 14:24:31.477239 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.477204    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee226b25-ee24-478c-ac2f-122773a08453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-mnbtf\" (UID: \"ee226b25-ee24-478c-ac2f-122773a08453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"
Jul 17 14:24:31.578761 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.578723    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee226b25-ee24-478c-ac2f-122773a08453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-mnbtf\" (UID: \"ee226b25-ee24-478c-ac2f-122773a08453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"
Jul 17 14:24:31.579378 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.579361    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee226b25-ee24-478c-ac2f-122773a08453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-mnbtf\" (UID: \"ee226b25-ee24-478c-ac2f-122773a08453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"
Jul 17 14:24:31.645604 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.645535    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"
Jul 17 14:24:31.784467 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.784443    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"]
Jul 17 14:24:31.786563 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:24:31.786530    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee226b25_ee24_478c_ac2f_122773a08453.slice/crio-b9bb678c3f8b8a90f85d8d57682ad3cabb606250b580790099bd29ac4110b134 WatchSource:0}: Error finding container b9bb678c3f8b8a90f85d8d57682ad3cabb606250b580790099bd29ac4110b134: Status 404 returned error can't find the container with id b9bb678c3f8b8a90f85d8d57682ad3cabb606250b580790099bd29ac4110b134
Jul 17 14:24:31.868097 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:31.868066    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf" event={"ID":"ee226b25-ee24-478c-ac2f-122773a08453","Type":"ContainerStarted","Data":"b9bb678c3f8b8a90f85d8d57682ad3cabb606250b580790099bd29ac4110b134"}
Jul 17 14:24:32.874572 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.874475    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee226b25-ee24-478c-ac2f-122773a08453" containerID="7555881ccb758ce5d4428dc51f4f1da5a6ae0c6681230d17834b3e68be69596d" exitCode=7
Jul 17 14:24:32.874572 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.874557    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf" event={"ID":"ee226b25-ee24-478c-ac2f-122773a08453","Type":"ContainerDied","Data":"7555881ccb758ce5d4428dc51f4f1da5a6ae0c6681230d17834b3e68be69596d"}
Jul 17 14:24:32.874983 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.874701    2561 scope.go:117] "RemoveContainer" containerID="7555881ccb758ce5d4428dc51f4f1da5a6ae0c6681230d17834b3e68be69596d"
Jul 17 14:24:32.931182 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.931155    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"]
Jul 17 14:24:32.931579 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.931567    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerName="revoke-keys"
Jul 17 14:24:32.931627 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.931581    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerName="revoke-keys"
Jul 17 14:24:32.931627 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.931590    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0db679d-588b-479e-810d-ecaac987eb82" containerName="revoke-keys"
Jul 17 14:24:32.931627 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.931596    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0db679d-588b-479e-810d-ecaac987eb82" containerName="revoke-keys"
Jul 17 14:24:32.931723 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.931675    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9961846-7954-4a71-ad88-4d8ba40a3069" containerName="revoke-keys"
Jul 17 14:24:32.935923 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.935908    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"
Jul 17 14:24:32.942544 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:32.942515    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"]
Jul 17 14:24:33.094372 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.094339    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/558c765e-d770-455b-bedb-8a3400cee707-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8\" (UID: \"558c765e-d770-455b-bedb-8a3400cee707\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"
Jul 17 14:24:33.195581 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.195547    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/558c765e-d770-455b-bedb-8a3400cee707-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8\" (UID: \"558c765e-d770-455b-bedb-8a3400cee707\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"
Jul 17 14:24:33.196171 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.196153    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/558c765e-d770-455b-bedb-8a3400cee707-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8\" (UID: \"558c765e-d770-455b-bedb-8a3400cee707\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"
Jul 17 14:24:33.248058 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.248034    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"
Jul 17 14:24:33.394854 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.394784    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"]
Jul 17 14:24:33.397230 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:24:33.397202    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558c765e_d770_455b_bedb_8a3400cee707.slice/crio-484b3b79d09f53666a4e6ba4ed5b3c129e21c588a9c40988767aa7ddf1df7f6b WatchSource:0}: Error finding container 484b3b79d09f53666a4e6ba4ed5b3c129e21c588a9c40988767aa7ddf1df7f6b: Status 404 returned error can't find the container with id 484b3b79d09f53666a4e6ba4ed5b3c129e21c588a9c40988767aa7ddf1df7f6b
Jul 17 14:24:33.881023 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.880969    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8" event={"ID":"558c765e-d770-455b-bedb-8a3400cee707","Type":"ContainerStarted","Data":"ed5efb7e37173e9474fd5458f85cfe96ee283b893aff5f1634a21622677da321"}
Jul 17 14:24:33.881459 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.881045    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8" event={"ID":"558c765e-d770-455b-bedb-8a3400cee707","Type":"ContainerStarted","Data":"484b3b79d09f53666a4e6ba4ed5b3c129e21c588a9c40988767aa7ddf1df7f6b"}
Jul 17 14:24:33.882766 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.882742    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee226b25-ee24-478c-ac2f-122773a08453" containerID="b121bd91926550b6465746bc6f0f043969597ab28d5def230d68f23632d311b4" exitCode=7
Jul 17 14:24:33.882879 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.882823    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf" event={"ID":"ee226b25-ee24-478c-ac2f-122773a08453","Type":"ContainerDied","Data":"b121bd91926550b6465746bc6f0f043969597ab28d5def230d68f23632d311b4"}
Jul 17 14:24:33.882923 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.882882    2561 scope.go:117] "RemoveContainer" containerID="7555881ccb758ce5d4428dc51f4f1da5a6ae0c6681230d17834b3e68be69596d"
Jul 17 14:24:33.882998 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.882984    2561 scope.go:117] "RemoveContainer" containerID="b121bd91926550b6465746bc6f0f043969597ab28d5def230d68f23632d311b4"
Jul 17 14:24:33.883239 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:24:33.883221    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-mnbtf_odh-ai-gateway-infra(ee226b25-ee24-478c-ac2f-122773a08453)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf" podUID="ee226b25-ee24-478c-ac2f-122773a08453"
Jul 17 14:24:33.898591 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:33.898549    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8" podStartSLOduration=1.898534304 podStartE2EDuration="1.898534304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:24:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:24:33.896655591 +0000 UTC m=+2359.603102427" watchObservedRunningTime="2026-07-17 14:24:33.898534304 +0000 UTC m=+2359.604981137"
Jul 17 14:24:34.897096 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:34.897058    2561 scope.go:117] "RemoveContainer" containerID="b121bd91926550b6465746bc6f0f043969597ab28d5def230d68f23632d311b4"
Jul 17 14:24:34.897539 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:24:34.897279    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-mnbtf_odh-ai-gateway-infra(ee226b25-ee24-478c-ac2f-122773a08453)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf" podUID="ee226b25-ee24-478c-ac2f-122773a08453"
Jul 17 14:24:34.898344 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:34.898323    2561 generic.go:356] "Generic (PLEG): container finished" podID="558c765e-d770-455b-bedb-8a3400cee707" containerID="ed5efb7e37173e9474fd5458f85cfe96ee283b893aff5f1634a21622677da321" exitCode=6
Jul 17 14:24:34.898449 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:34.898361    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8" event={"ID":"558c765e-d770-455b-bedb-8a3400cee707","Type":"ContainerDied","Data":"ed5efb7e37173e9474fd5458f85cfe96ee283b893aff5f1634a21622677da321"}
Jul 17 14:24:34.898560 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:34.898549    2561 scope.go:117] "RemoveContainer" containerID="ed5efb7e37173e9474fd5458f85cfe96ee283b893aff5f1634a21622677da321"
Jul 17 14:24:35.905436 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:35.905346    2561 generic.go:356] "Generic (PLEG): container finished" podID="558c765e-d770-455b-bedb-8a3400cee707" containerID="87040c295fd9dbe62cfb2df230f3d1a93cd3460e0e7a491eccee5077591c41a2" exitCode=6
Jul 17 14:24:35.905871 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:35.905445    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8" event={"ID":"558c765e-d770-455b-bedb-8a3400cee707","Type":"ContainerDied","Data":"87040c295fd9dbe62cfb2df230f3d1a93cd3460e0e7a491eccee5077591c41a2"}
Jul 17 14:24:35.905871 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:35.905510    2561 scope.go:117] "RemoveContainer" containerID="ed5efb7e37173e9474fd5458f85cfe96ee283b893aff5f1634a21622677da321"
Jul 17 14:24:35.905871 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:35.905611    2561 scope.go:117] "RemoveContainer" containerID="87040c295fd9dbe62cfb2df230f3d1a93cd3460e0e7a491eccee5077591c41a2"
Jul 17 14:24:35.905871 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:24:35.905833    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8_odh-ai-gateway-infra(558c765e-d770-455b-bedb-8a3400cee707)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8" podUID="558c765e-d770-455b-bedb-8a3400cee707"
Jul 17 14:24:36.912909 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:36.912882    2561 scope.go:117] "RemoveContainer" containerID="87040c295fd9dbe62cfb2df230f3d1a93cd3460e0e7a491eccee5077591c41a2"
Jul 17 14:24:36.913379 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:24:36.913192    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8_odh-ai-gateway-infra(558c765e-d770-455b-bedb-8a3400cee707)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8" podUID="558c765e-d770-455b-bedb-8a3400cee707"
Jul 17 14:24:42.045408 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.045370    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"]
Jul 17 14:24:42.050222 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.050205    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"
Jul 17 14:24:42.057193 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.057164    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"]
Jul 17 14:24:42.185226 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.185194    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1a3ae75-04ca-438a-8176-9ccd884d22de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg\" (UID: \"b1a3ae75-04ca-438a-8176-9ccd884d22de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"
Jul 17 14:24:42.286447 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.286413    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1a3ae75-04ca-438a-8176-9ccd884d22de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg\" (UID: \"b1a3ae75-04ca-438a-8176-9ccd884d22de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"
Jul 17 14:24:42.287120 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.287096    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1a3ae75-04ca-438a-8176-9ccd884d22de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg\" (UID: \"b1a3ae75-04ca-438a-8176-9ccd884d22de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"
Jul 17 14:24:42.362567 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.362495    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"
Jul 17 14:24:42.501178 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.501155    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"]
Jul 17 14:24:42.504167 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:24:42.504128    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a3ae75_04ca_438a_8176_9ccd884d22de.slice/crio-53330ec2c8c849575420b36b248ff0a5f4f197618ca110fd98abf635f55bc73a WatchSource:0}: Error finding container 53330ec2c8c849575420b36b248ff0a5f4f197618ca110fd98abf635f55bc73a: Status 404 returned error can't find the container with id 53330ec2c8c849575420b36b248ff0a5f4f197618ca110fd98abf635f55bc73a
Jul 17 14:24:42.946163 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.946130    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" event={"ID":"b1a3ae75-04ca-438a-8176-9ccd884d22de","Type":"ContainerStarted","Data":"b5320de6faf23b0ce32a1d99929e89e263558ff4bbe17266f77dbca9712a9b6e"}
Jul 17 14:24:42.946323 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.946182    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" event={"ID":"b1a3ae75-04ca-438a-8176-9ccd884d22de","Type":"ContainerStarted","Data":"53330ec2c8c849575420b36b248ff0a5f4f197618ca110fd98abf635f55bc73a"}
Jul 17 14:24:42.964079 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:42.964033    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" podStartSLOduration=0.963997802 podStartE2EDuration="963.997802ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:24:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:24:42.962719658 +0000 UTC m=+2368.669166502" watchObservedRunningTime="2026-07-17 14:24:42.963997802 +0000 UTC m=+2368.670444635"
Jul 17 14:24:43.953259 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:43.953227    2561 generic.go:356] "Generic (PLEG): container finished" podID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerID="b5320de6faf23b0ce32a1d99929e89e263558ff4bbe17266f77dbca9712a9b6e" exitCode=6
Jul 17 14:24:43.953629 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:43.953314    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" event={"ID":"b1a3ae75-04ca-438a-8176-9ccd884d22de","Type":"ContainerDied","Data":"b5320de6faf23b0ce32a1d99929e89e263558ff4bbe17266f77dbca9712a9b6e"}
Jul 17 14:24:43.953629 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:43.953457    2561 scope.go:117] "RemoveContainer" containerID="b5320de6faf23b0ce32a1d99929e89e263558ff4bbe17266f77dbca9712a9b6e"
Jul 17 14:24:44.960614 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:44.960581    2561 generic.go:356] "Generic (PLEG): container finished" podID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerID="31cc50b0b1d04a28ac5194dffeaa7e8d1101e57a2a5ffe1df91358284b300166" exitCode=6
Jul 17 14:24:44.961019 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:44.960674    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" event={"ID":"b1a3ae75-04ca-438a-8176-9ccd884d22de","Type":"ContainerDied","Data":"31cc50b0b1d04a28ac5194dffeaa7e8d1101e57a2a5ffe1df91358284b300166"}
Jul 17 14:24:44.961019 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:44.960735    2561 scope.go:117] "RemoveContainer" containerID="b5320de6faf23b0ce32a1d99929e89e263558ff4bbe17266f77dbca9712a9b6e"
Jul 17 14:24:44.961019 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:44.960842    2561 scope.go:117] "RemoveContainer" containerID="31cc50b0b1d04a28ac5194dffeaa7e8d1101e57a2a5ffe1df91358284b300166"
Jul 17 14:24:44.961190 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:24:44.961134    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg_odh-ai-gateway-infra(b1a3ae75-04ca-438a-8176-9ccd884d22de)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de"
Jul 17 14:24:45.974399 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:45.974369    2561 scope.go:117] "RemoveContainer" containerID="31cc50b0b1d04a28ac5194dffeaa7e8d1101e57a2a5ffe1df91358284b300166"
Jul 17 14:24:45.974774 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:24:45.974576    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg_odh-ai-gateway-infra(b1a3ae75-04ca-438a-8176-9ccd884d22de)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de"
Jul 17 14:24:49.765540 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:49.765516    2561 scope.go:117] "RemoveContainer" containerID="b121bd91926550b6465746bc6f0f043969597ab28d5def230d68f23632d311b4"
Jul 17 14:24:49.999275 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:49.999237    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf" event={"ID":"ee226b25-ee24-478c-ac2f-122773a08453","Type":"ContainerStarted","Data":"16d897f089f4a2f53300f63ba70f09ec87ab49d9cb7131943b3d0ca42b79e0d5"}
Jul 17 14:24:50.015859 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:50.015756    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf" podStartSLOduration=19.015741837 podStartE2EDuration="19.015741837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:24:50.014613657 +0000 UTC m=+2375.721060512" watchObservedRunningTime="2026-07-17 14:24:50.015741837 +0000 UTC m=+2375.722188681"
Jul 17 14:24:50.766422 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:50.766392    2561 scope.go:117] "RemoveContainer" containerID="87040c295fd9dbe62cfb2df230f3d1a93cd3460e0e7a491eccee5077591c41a2"
Jul 17 14:24:51.006424 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:51.006391    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee226b25-ee24-478c-ac2f-122773a08453" containerID="16d897f089f4a2f53300f63ba70f09ec87ab49d9cb7131943b3d0ca42b79e0d5" exitCode=7
Jul 17 14:24:51.006582 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:51.006460    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf" event={"ID":"ee226b25-ee24-478c-ac2f-122773a08453","Type":"ContainerDied","Data":"16d897f089f4a2f53300f63ba70f09ec87ab49d9cb7131943b3d0ca42b79e0d5"}
Jul 17 14:24:51.006582 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:51.006526    2561 scope.go:117] "RemoveContainer" containerID="b121bd91926550b6465746bc6f0f043969597ab28d5def230d68f23632d311b4"
Jul 17 14:24:51.006863 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:51.006843    2561 scope.go:117] "RemoveContainer" containerID="16d897f089f4a2f53300f63ba70f09ec87ab49d9cb7131943b3d0ca42b79e0d5"
Jul 17 14:24:51.007114 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:24:51.007095    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-mnbtf_odh-ai-gateway-infra(ee226b25-ee24-478c-ac2f-122773a08453)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf" podUID="ee226b25-ee24-478c-ac2f-122773a08453"
Jul 17 14:24:51.028403 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:51.028376    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"]
Jul 17 14:24:52.013933 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:52.013900    2561 generic.go:356] "Generic (PLEG): container finished" podID="558c765e-d770-455b-bedb-8a3400cee707" containerID="98db8fb80e75ea545175c7c24fbb83256a760efd152a58aaa088f037b11044ab" exitCode=6
Jul 17 14:24:52.014394 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:52.013979    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8" event={"ID":"558c765e-d770-455b-bedb-8a3400cee707","Type":"ContainerDied","Data":"98db8fb80e75ea545175c7c24fbb83256a760efd152a58aaa088f037b11044ab"}
Jul 17 14:24:52.014394 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:52.014056    2561 scope.go:117] "RemoveContainer" containerID="87040c295fd9dbe62cfb2df230f3d1a93cd3460e0e7a491eccee5077591c41a2"
Jul 17 14:24:52.014394 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:52.014316    2561 scope.go:117] "RemoveContainer" containerID="98db8fb80e75ea545175c7c24fbb83256a760efd152a58aaa088f037b11044ab"
Jul 17 14:24:52.014593 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:24:52.014571    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8_odh-ai-gateway-infra(558c765e-d770-455b-bedb-8a3400cee707)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8" podUID="558c765e-d770-455b-bedb-8a3400cee707"
Jul 17 14:24:52.159478 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:52.159453    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"
Jul 17 14:24:52.177049 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:52.176995    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee226b25-ee24-478c-ac2f-122773a08453-maas-api-service-ca\") pod \"ee226b25-ee24-478c-ac2f-122773a08453\" (UID: \"ee226b25-ee24-478c-ac2f-122773a08453\") "
Jul 17 14:24:52.177360 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:52.177334    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee226b25-ee24-478c-ac2f-122773a08453-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ee226b25-ee24-478c-ac2f-122773a08453" (UID: "ee226b25-ee24-478c-ac2f-122773a08453"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:24:52.177519 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:52.177489    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee226b25-ee24-478c-ac2f-122773a08453-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:24:53.024833 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:53.024804    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"
Jul 17 14:24:53.025311 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:53.024802    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf" event={"ID":"ee226b25-ee24-478c-ac2f-122773a08453","Type":"ContainerDied","Data":"b9bb678c3f8b8a90f85d8d57682ad3cabb606250b580790099bd29ac4110b134"}
Jul 17 14:24:53.025311 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:53.024936    2561 scope.go:117] "RemoveContainer" containerID="16d897f089f4a2f53300f63ba70f09ec87ab49d9cb7131943b3d0ca42b79e0d5"
Jul 17 14:24:53.040874 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:53.040851    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"]
Jul 17 14:24:53.045021 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:53.044987    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"]
Jul 17 14:24:53.053736 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:53.053715    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-mnbtf"]
Jul 17 14:24:53.178727 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:53.178706    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"
Jul 17 14:24:53.289824 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:53.289736    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/558c765e-d770-455b-bedb-8a3400cee707-maas-api-service-ca\") pod \"558c765e-d770-455b-bedb-8a3400cee707\" (UID: \"558c765e-d770-455b-bedb-8a3400cee707\") "
Jul 17 14:24:53.290169 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:53.290143    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558c765e-d770-455b-bedb-8a3400cee707-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "558c765e-d770-455b-bedb-8a3400cee707" (UID: "558c765e-d770-455b-bedb-8a3400cee707"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:24:53.391442 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:53.391392    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/558c765e-d770-455b-bedb-8a3400cee707-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:24:54.032806 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:54.032774    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8" event={"ID":"558c765e-d770-455b-bedb-8a3400cee707","Type":"ContainerDied","Data":"484b3b79d09f53666a4e6ba4ed5b3c129e21c588a9c40988767aa7ddf1df7f6b"}
Jul 17 14:24:54.032806 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:54.032804    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"
Jul 17 14:24:54.033305 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:54.032831    2561 scope.go:117] "RemoveContainer" containerID="98db8fb80e75ea545175c7c24fbb83256a760efd152a58aaa088f037b11044ab"
Jul 17 14:24:54.059934 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:54.059906    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"]
Jul 17 14:24:54.067452 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:54.067429    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-rtsh8"]
Jul 17 14:24:54.773774 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:54.773740    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="558c765e-d770-455b-bedb-8a3400cee707" path="/var/lib/kubelet/pods/558c765e-d770-455b-bedb-8a3400cee707/volumes"
Jul 17 14:24:54.774134 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:54.774120    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee226b25-ee24-478c-ac2f-122773a08453" path="/var/lib/kubelet/pods/ee226b25-ee24-478c-ac2f-122773a08453/volumes"
Jul 17 14:24:58.766265 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:58.766229    2561 scope.go:117] "RemoveContainer" containerID="31cc50b0b1d04a28ac5194dffeaa7e8d1101e57a2a5ffe1df91358284b300166"
Jul 17 14:24:59.062753 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:24:59.062723    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" event={"ID":"b1a3ae75-04ca-438a-8176-9ccd884d22de","Type":"ContainerStarted","Data":"7247a30b57a2a433844e7d9a425902c94576b7e592fc3733a5511bc2f40aa767"}
Jul 17 14:25:00.070114 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:00.070081    2561 generic.go:356] "Generic (PLEG): container finished" podID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerID="7247a30b57a2a433844e7d9a425902c94576b7e592fc3733a5511bc2f40aa767" exitCode=6
Jul 17 14:25:00.070579 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:00.070158    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" event={"ID":"b1a3ae75-04ca-438a-8176-9ccd884d22de","Type":"ContainerDied","Data":"7247a30b57a2a433844e7d9a425902c94576b7e592fc3733a5511bc2f40aa767"}
Jul 17 14:25:00.070579 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:00.070216    2561 scope.go:117] "RemoveContainer" containerID="31cc50b0b1d04a28ac5194dffeaa7e8d1101e57a2a5ffe1df91358284b300166"
Jul 17 14:25:00.070579 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:00.070557    2561 scope.go:117] "RemoveContainer" containerID="7247a30b57a2a433844e7d9a425902c94576b7e592fc3733a5511bc2f40aa767"
Jul 17 14:25:00.070810 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:00.070787    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg_odh-ai-gateway-infra(b1a3ae75-04ca-438a-8176-9ccd884d22de)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de"
Jul 17 14:25:00.097477 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:00.097453    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"]
Jul 17 14:25:01.223403 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:01.223376    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"
Jul 17 14:25:01.258937 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:01.258904    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"]
Jul 17 14:25:01.259133 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:01.259110    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy" containerID="cri-o://4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5" gracePeriod=30
Jul 17 14:25:01.272562 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:01.272537    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1a3ae75-04ca-438a-8176-9ccd884d22de-maas-api-service-ca\") pod \"b1a3ae75-04ca-438a-8176-9ccd884d22de\" (UID: \"b1a3ae75-04ca-438a-8176-9ccd884d22de\") "
Jul 17 14:25:01.272893 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:01.272870    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1a3ae75-04ca-438a-8176-9ccd884d22de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b1a3ae75-04ca-438a-8176-9ccd884d22de" (UID: "b1a3ae75-04ca-438a-8176-9ccd884d22de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:25:01.373938 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:01.373859    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1a3ae75-04ca-438a-8176-9ccd884d22de-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:02.083358 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:02.083316    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"
Jul 17 14:25:02.083358 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:02.083350    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg" event={"ID":"b1a3ae75-04ca-438a-8176-9ccd884d22de","Type":"ContainerDied","Data":"53330ec2c8c849575420b36b248ff0a5f4f197618ca110fd98abf635f55bc73a"}
Jul 17 14:25:02.083603 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:02.083412    2561 scope.go:117] "RemoveContainer" containerID="7247a30b57a2a433844e7d9a425902c94576b7e592fc3733a5511bc2f40aa767"
Jul 17 14:25:02.109086 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:02.109061    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"]
Jul 17 14:25:02.116138 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:02.116115    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-bhkvg"]
Jul 17 14:25:02.772130 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:02.772097    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de" path="/var/lib/kubelet/pods/b1a3ae75-04ca-438a-8176-9ccd884d22de/volumes"
Jul 17 14:25:06.511887 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.511864    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:25:06.623432 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.623397    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-certs\") pod \"f95f76fb-406d-4c59-a655-ae391d5952e3\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") "
Jul 17 14:25:06.623662 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.623465    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95f76fb-406d-4c59-a655-ae391d5952e3-istiod-ca-cert\") pod \"f95f76fb-406d-4c59-a655-ae391d5952e3\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") "
Jul 17 14:25:06.623662 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.623496    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-data\") pod \"f95f76fb-406d-4c59-a655-ae391d5952e3\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") "
Jul 17 14:25:06.623662 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.623555    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-token\") pod \"f95f76fb-406d-4c59-a655-ae391d5952e3\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") "
Jul 17 14:25:06.623662 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.623580    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-socket\") pod \"f95f76fb-406d-4c59-a655-ae391d5952e3\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") "
Jul 17 14:25:06.623662 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.623647    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-podinfo\") pod \"f95f76fb-406d-4c59-a655-ae391d5952e3\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") "
Jul 17 14:25:06.623977 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.623687    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzcsx\" (UniqueName: \"kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-kube-api-access-wzcsx\") pod \"f95f76fb-406d-4c59-a655-ae391d5952e3\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") "
Jul 17 14:25:06.623977 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.623734    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-credential-socket\") pod \"f95f76fb-406d-4c59-a655-ae391d5952e3\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") "
Jul 17 14:25:06.623977 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.623780    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-envoy\") pod \"f95f76fb-406d-4c59-a655-ae391d5952e3\" (UID: \"f95f76fb-406d-4c59-a655-ae391d5952e3\") "
Jul 17 14:25:06.624993 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.624427    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f95f76fb-406d-4c59-a655-ae391d5952e3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f95f76fb-406d-4c59-a655-ae391d5952e3" (UID: "f95f76fb-406d-4c59-a655-ae391d5952e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:25:06.624993 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.624909    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f95f76fb-406d-4c59-a655-ae391d5952e3" (UID: "f95f76fb-406d-4c59-a655-ae391d5952e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:25:06.624993 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.624968    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f95f76fb-406d-4c59-a655-ae391d5952e3" (UID: "f95f76fb-406d-4c59-a655-ae391d5952e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:25:06.625598 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.625573    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-data" (OuterVolumeSpecName: "istio-data") pod "f95f76fb-406d-4c59-a655-ae391d5952e3" (UID: "f95f76fb-406d-4c59-a655-ae391d5952e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:25:06.625691 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.625644    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f95f76fb-406d-4c59-a655-ae391d5952e3" (UID: "f95f76fb-406d-4c59-a655-ae391d5952e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:25:06.627478 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.627451    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f95f76fb-406d-4c59-a655-ae391d5952e3" (UID: "f95f76fb-406d-4c59-a655-ae391d5952e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:25:06.629376 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.629349    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-kube-api-access-wzcsx" (OuterVolumeSpecName: "kube-api-access-wzcsx") pod "f95f76fb-406d-4c59-a655-ae391d5952e3" (UID: "f95f76fb-406d-4c59-a655-ae391d5952e3"). InnerVolumeSpecName "kube-api-access-wzcsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:25:06.630283 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.630257    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f95f76fb-406d-4c59-a655-ae391d5952e3" (UID: "f95f76fb-406d-4c59-a655-ae391d5952e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:25:06.630400 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.630294    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-token" (OuterVolumeSpecName: "istio-token") pod "f95f76fb-406d-4c59-a655-ae391d5952e3" (UID: "f95f76fb-406d-4c59-a655-ae391d5952e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:25:06.725501 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.725469    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:06.725501 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.725503    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:06.725737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.725517    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:06.725737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.725527    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzcsx\" (UniqueName: \"kubernetes.io/projected/f95f76fb-406d-4c59-a655-ae391d5952e3-kube-api-access-wzcsx\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:06.725737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.725535    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:06.725737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.725545    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:06.725737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.725553    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:06.725737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.725560    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95f76fb-406d-4c59-a655-ae391d5952e3-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:06.725737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:06.725568    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95f76fb-406d-4c59-a655-ae391d5952e3-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:07.111873 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.111836    2561 generic.go:356] "Generic (PLEG): container finished" podID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerID="4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5" exitCode=0
Jul 17 14:25:07.112059 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.111894    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"
Jul 17 14:25:07.112059 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.111921    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" event={"ID":"f95f76fb-406d-4c59-a655-ae391d5952e3","Type":"ContainerDied","Data":"4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5"}
Jul 17 14:25:07.112059 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.111975    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p" event={"ID":"f95f76fb-406d-4c59-a655-ae391d5952e3","Type":"ContainerDied","Data":"b7113a6322dd8147bb8a06033e85113bf7f155d9cb47e89dea404ed4677d20d3"}
Jul 17 14:25:07.112059 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.111999    2561 scope.go:117] "RemoveContainer" containerID="4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5"
Jul 17 14:25:07.125079 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.125059    2561 scope.go:117] "RemoveContainer" containerID="c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364"
Jul 17 14:25:07.133210 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.133193    2561 scope.go:117] "RemoveContainer" containerID="4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5"
Jul 17 14:25:07.133476 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:07.133455    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5\": container with ID starting with 4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5 not found: ID does not exist" containerID="4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5"
Jul 17 14:25:07.133538 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.133487    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5"} err="failed to get container status \"4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5\": rpc error: code = NotFound desc = could not find container \"4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5\": container with ID starting with 4b87f26c6ccca684662f04f5b774d9a4dc5c0fa49eb93c046b6de15f79dc94a5 not found: ID does not exist"
Jul 17 14:25:07.133538 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.133504    2561 scope.go:117] "RemoveContainer" containerID="c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364"
Jul 17 14:25:07.133743 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:07.133720    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364\": container with ID starting with c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364 not found: ID does not exist" containerID="c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364"
Jul 17 14:25:07.133784 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.133754    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364"} err="failed to get container status \"c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364\": rpc error: code = NotFound desc = could not find container \"c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364\": container with ID starting with c784cfa517582bd845966ab85d8833ca9a124167775faeb10979b1424d50e364 not found: ID does not exist"
Jul 17 14:25:07.138123 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.138104    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"]
Jul 17 14:25:07.144193 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:07.144171    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d7874222-openshift-default-86cc7f48dc-mh97p"]
Jul 17 14:25:08.772667 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:08.772633    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" path="/var/lib/kubelet/pods/f95f76fb-406d-4c59-a655-ae391d5952e3/volumes"
Jul 17 14:25:13.183849 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.183811    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"]
Jul 17 14:25:13.184498 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184471    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="558c765e-d770-455b-bedb-8a3400cee707" containerName="revoke-keys"
Jul 17 14:25:13.184498 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184499    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="558c765e-d770-455b-bedb-8a3400cee707" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184528    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184534    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184548    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184557    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184575    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184583    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184594    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee226b25-ee24-478c-ac2f-122773a08453" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184602    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee226b25-ee24-478c-ac2f-122773a08453" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184611    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184618    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184632    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184639    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184647    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee226b25-ee24-478c-ac2f-122773a08453" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184652    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee226b25-ee24-478c-ac2f-122773a08453" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184657    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="558c765e-d770-455b-bedb-8a3400cee707" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184661    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="558c765e-d770-455b-bedb-8a3400cee707" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184672    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee226b25-ee24-478c-ac2f-122773a08453" containerName="revoke-keys"
Jul 17 14:25:13.184722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184680    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee226b25-ee24-478c-ac2f-122773a08453" containerName="revoke-keys"
Jul 17 14:25:13.185380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184759    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerName="revoke-keys"
Jul 17 14:25:13.185380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184767    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy"
Jul 17 14:25:13.185380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184773    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee226b25-ee24-478c-ac2f-122773a08453" containerName="revoke-keys"
Jul 17 14:25:13.185380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184782    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="558c765e-d770-455b-bedb-8a3400cee707" containerName="revoke-keys"
Jul 17 14:25:13.185380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184791    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee226b25-ee24-478c-ac2f-122773a08453" containerName="revoke-keys"
Jul 17 14:25:13.185380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184797    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="558c765e-d770-455b-bedb-8a3400cee707" containerName="revoke-keys"
Jul 17 14:25:13.185380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184803    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="558c765e-d770-455b-bedb-8a3400cee707" containerName="revoke-keys"
Jul 17 14:25:13.185380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.184811    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerName="revoke-keys"
Jul 17 14:25:13.189433 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.189417    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.192663 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.192643    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-73fe1bea-openshift-default-dockercfg-8b6fs\""
Jul 17 14:25:13.202645 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.202624    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"]
Jul 17 14:25:13.294027 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.293969    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-certs\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.294222 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.294110    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-data\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.294222 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.294139    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-socket\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.294222 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.294160    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-envoy\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.294222 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.294217    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-podinfo\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.294426 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.294258    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-credential-socket\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.294426 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.294286    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istiod-ca-cert\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.294426 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.294310    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-token\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.294426 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.294381    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmb5g\" (UniqueName: \"kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-kube-api-access-rmb5g\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.395544 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.395510    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-credential-socket\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.395544 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.395549    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istiod-ca-cert\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.395744 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.395676    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-token\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.395744 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.395730    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmb5g\" (UniqueName: \"kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-kube-api-access-rmb5g\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.395827 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.395779    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-certs\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.395886 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.395826    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-data\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.395886 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.395873    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-socket\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.395987 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.395895    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-envoy\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.395987 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.395926    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-credential-socket\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.395987 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.395942    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-podinfo\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.396218 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.396199    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-certs\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.396308 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.396290    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istiod-ca-cert\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.396348 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.396329    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-data\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.396552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.396526    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-socket\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.398440 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.398420    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-envoy\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.398555 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.398485    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-podinfo\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.403795 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.403772    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-token\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.404185 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.404164    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmb5g\" (UniqueName: \"kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-kube-api-access-rmb5g\") pod \"e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") " pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.500604 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.500561    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:13.656673 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.656648    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"]
Jul 17 14:25:13.658762 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:25:13.658727    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac86132f_6c04_4114_a3c6_f23ff4ea43ee.slice/crio-d2f1fc5e4734d205f5280223424387cd817ced6a8cdeed4f97be022c646780a5 WatchSource:0}: Error finding container d2f1fc5e4734d205f5280223424387cd817ced6a8cdeed4f97be022c646780a5: Status 404 returned error can't find the container with id d2f1fc5e4734d205f5280223424387cd817ced6a8cdeed4f97be022c646780a5
Jul 17 14:25:13.661328 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.661295    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:25:13.661441 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.661378    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:25:13.661441 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:13.661422    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:25:14.152111 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:14.152081    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" event={"ID":"ac86132f-6c04-4114-a3c6-f23ff4ea43ee","Type":"ContainerStarted","Data":"8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb"}
Jul 17 14:25:14.152276 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:14.152129    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" event={"ID":"ac86132f-6c04-4114-a3c6-f23ff4ea43ee","Type":"ContainerStarted","Data":"d2f1fc5e4734d205f5280223424387cd817ced6a8cdeed4f97be022c646780a5"}
Jul 17 14:25:14.174312 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:14.174257    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" podStartSLOduration=1.174242974 podStartE2EDuration="1.174242974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:25:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:25:14.172255632 +0000 UTC m=+2399.878702478" watchObservedRunningTime="2026-07-17 14:25:14.174242974 +0000 UTC m=+2399.880689809"
Jul 17 14:25:14.501564 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:14.501530    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:14.506164 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:14.506138    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:15.158095 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:15.158055    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:15.158951 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:15.158932    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:18.733611 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.733574    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"]
Jul 17 14:25:18.734174 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.734150    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="558c765e-d770-455b-bedb-8a3400cee707" containerName="revoke-keys"
Jul 17 14:25:18.734343 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.734324    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="558c765e-d770-455b-bedb-8a3400cee707" containerName="revoke-keys"
Jul 17 14:25:18.734470 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.734459    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1a3ae75-04ca-438a-8176-9ccd884d22de" containerName="revoke-keys"
Jul 17 14:25:18.734523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.734472    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f95f76fb-406d-4c59-a655-ae391d5952e3" containerName="istio-proxy"
Jul 17 14:25:18.734523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.734481    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee226b25-ee24-478c-ac2f-122773a08453" containerName="revoke-keys"
Jul 17 14:25:18.739063 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.739047    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"
Jul 17 14:25:18.741881 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.741839    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:25:18.742766 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.742744    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:25:18.744935 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.744912    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"]
Jul 17 14:25:18.849341 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.849308    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac50f1ba-3891-4f98-a5b1-e788527a19f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt\" (UID: \"ac50f1ba-3891-4f98-a5b1-e788527a19f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"
Jul 17 14:25:18.950692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.950659    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac50f1ba-3891-4f98-a5b1-e788527a19f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt\" (UID: \"ac50f1ba-3891-4f98-a5b1-e788527a19f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"
Jul 17 14:25:18.951306 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:18.951286    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac50f1ba-3891-4f98-a5b1-e788527a19f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt\" (UID: \"ac50f1ba-3891-4f98-a5b1-e788527a19f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"
Jul 17 14:25:19.051512 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:19.051427    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"
Jul 17 14:25:19.251098 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:19.244522    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"]
Jul 17 14:25:20.200870 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:20.200839    2561 generic.go:356] "Generic (PLEG): container finished" podID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerID="12c928172bc39b7880499fd5af31c7c705d61f150c3ddafa1949d75c8848754a" exitCode=7
Jul 17 14:25:20.201245 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:20.200897    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt" event={"ID":"ac50f1ba-3891-4f98-a5b1-e788527a19f3","Type":"ContainerDied","Data":"12c928172bc39b7880499fd5af31c7c705d61f150c3ddafa1949d75c8848754a"}
Jul 17 14:25:20.201245 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:20.200950    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt" event={"ID":"ac50f1ba-3891-4f98-a5b1-e788527a19f3","Type":"ContainerStarted","Data":"49035af85d7830201bad25eedebedc360789c12d4a4b57f49f95b0bdfd1bc199"}
Jul 17 14:25:20.201245 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:20.201125    2561 scope.go:117] "RemoveContainer" containerID="12c928172bc39b7880499fd5af31c7c705d61f150c3ddafa1949d75c8848754a"
Jul 17 14:25:21.208084 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:21.208053    2561 generic.go:356] "Generic (PLEG): container finished" podID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerID="6783ed2dc149784cb9b4860fcef759c8de7bc009c85c343f91770f4fc368e752" exitCode=7
Jul 17 14:25:21.208508 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:21.208131    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt" event={"ID":"ac50f1ba-3891-4f98-a5b1-e788527a19f3","Type":"ContainerDied","Data":"6783ed2dc149784cb9b4860fcef759c8de7bc009c85c343f91770f4fc368e752"}
Jul 17 14:25:21.208508 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:21.208188    2561 scope.go:117] "RemoveContainer" containerID="12c928172bc39b7880499fd5af31c7c705d61f150c3ddafa1949d75c8848754a"
Jul 17 14:25:21.208508 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:21.208358    2561 scope.go:117] "RemoveContainer" containerID="6783ed2dc149784cb9b4860fcef759c8de7bc009c85c343f91770f4fc368e752"
Jul 17 14:25:21.208679 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:21.208654    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt_odh-ai-gateway-infra(ac50f1ba-3891-4f98-a5b1-e788527a19f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3"
Jul 17 14:25:22.216336 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:22.216310    2561 scope.go:117] "RemoveContainer" containerID="6783ed2dc149784cb9b4860fcef759c8de7bc009c85c343f91770f4fc368e752"
Jul 17 14:25:22.216730 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:22.216511    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt_odh-ai-gateway-infra(ac50f1ba-3891-4f98-a5b1-e788527a19f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3"
Jul 17 14:25:26.243375 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:26.243341    2561 generic.go:356] "Generic (PLEG): container finished" podID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerID="8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb" exitCode=0
Jul 17 14:25:26.243779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:26.243414    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" event={"ID":"ac86132f-6c04-4114-a3c6-f23ff4ea43ee","Type":"ContainerDied","Data":"8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb"}
Jul 17 14:25:26.243824 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:26.243803    2561 scope.go:117] "RemoveContainer" containerID="8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb"
Jul 17 14:25:26.244830 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:26.244801    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:25:26.244927 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:26.244881    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:25:26.244995 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:26.244931    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:25:27.254922 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:27.254880    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" event={"ID":"ac86132f-6c04-4114-a3c6-f23ff4ea43ee","Type":"ContainerStarted","Data":"2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b"}
Jul 17 14:25:27.501328 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:27.501293    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:27.502588 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:27.502563    2561 patch_prober.go:28] interesting pod/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Jul 17 14:25:27.502704 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:27.502615    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Jul 17 14:25:28.501311 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:28.501274    2561 patch_prober.go:28] interesting pod/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Jul 17 14:25:28.501704 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:28.501327    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Jul 17 14:25:29.501090 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:29.501054    2561 patch_prober.go:28] interesting pod/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Jul 17 14:25:29.501271 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:29.501107    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Jul 17 14:25:30.158347 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:30.158315    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:30.501413 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:30.501382    2561 patch_prober.go:28] interesting pod/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Jul 17 14:25:30.501582 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:30.501435    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Jul 17 14:25:31.409135 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:31.409100    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"]
Jul 17 14:25:31.412185 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:31.412164    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"
Jul 17 14:25:31.422819 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:31.422753    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"]
Jul 17 14:25:31.458799 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:31.458768    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6768be86-6903-43a4-acce-22b9bb42eec3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-z5zkt\" (UID: \"6768be86-6903-43a4-acce-22b9bb42eec3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"
Jul 17 14:25:31.505298 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:31.505269    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:31.506247 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:31.506229    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:25:31.560319 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:31.560283    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6768be86-6903-43a4-acce-22b9bb42eec3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-z5zkt\" (UID: \"6768be86-6903-43a4-acce-22b9bb42eec3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"
Jul 17 14:25:31.561124 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:31.561100    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6768be86-6903-43a4-acce-22b9bb42eec3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-z5zkt\" (UID: \"6768be86-6903-43a4-acce-22b9bb42eec3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"
Jul 17 14:25:31.723204 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:31.723178    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"
Jul 17 14:25:32.066861 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:32.066835    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"]
Jul 17 14:25:32.068367 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:25:32.068329    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6768be86_6903_43a4_acce_22b9bb42eec3.slice/crio-0072017f5ed9c810e913ea8babc233ee849d3b8b76485fa92e49d5117676bc8b WatchSource:0}: Error finding container 0072017f5ed9c810e913ea8babc233ee849d3b8b76485fa92e49d5117676bc8b: Status 404 returned error can't find the container with id 0072017f5ed9c810e913ea8babc233ee849d3b8b76485fa92e49d5117676bc8b
Jul 17 14:25:32.281924 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:32.281883    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt" event={"ID":"6768be86-6903-43a4-acce-22b9bb42eec3","Type":"ContainerStarted","Data":"f6ba0aae2931dc207dc56ff381b950f760cc4a4f73e13686849db4092f4279bd"}
Jul 17 14:25:32.282070 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:32.281944    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt" event={"ID":"6768be86-6903-43a4-acce-22b9bb42eec3","Type":"ContainerStarted","Data":"0072017f5ed9c810e913ea8babc233ee849d3b8b76485fa92e49d5117676bc8b"}
Jul 17 14:25:32.299078 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:32.299035    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt" podStartSLOduration=1.299001147 podStartE2EDuration="1.299001147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:25:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:25:32.297098647 +0000 UTC m=+2418.003545481" watchObservedRunningTime="2026-07-17 14:25:32.299001147 +0000 UTC m=+2418.005447980"
Jul 17 14:25:32.977435 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:32.977402    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"]
Jul 17 14:25:32.980155 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:32.980136    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"
Jul 17 14:25:32.988674 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:32.988647    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"]
Jul 17 14:25:33.075898 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:33.075869    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/079ceb74-2221-43f4-934a-63744a10f0dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-77hb5\" (UID: \"079ceb74-2221-43f4-934a-63744a10f0dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"
Jul 17 14:25:33.177227 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:33.177201    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/079ceb74-2221-43f4-934a-63744a10f0dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-77hb5\" (UID: \"079ceb74-2221-43f4-934a-63744a10f0dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"
Jul 17 14:25:33.177786 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:33.177768    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/079ceb74-2221-43f4-934a-63744a10f0dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-77hb5\" (UID: \"079ceb74-2221-43f4-934a-63744a10f0dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"
Jul 17 14:25:33.289356 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:33.289290    2561 generic.go:356] "Generic (PLEG): container finished" podID="6768be86-6903-43a4-acce-22b9bb42eec3" containerID="f6ba0aae2931dc207dc56ff381b950f760cc4a4f73e13686849db4092f4279bd" exitCode=6
Jul 17 14:25:33.289479 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:33.289370    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt" event={"ID":"6768be86-6903-43a4-acce-22b9bb42eec3","Type":"ContainerDied","Data":"f6ba0aae2931dc207dc56ff381b950f760cc4a4f73e13686849db4092f4279bd"}
Jul 17 14:25:33.289553 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:33.289539    2561 scope.go:117] "RemoveContainer" containerID="f6ba0aae2931dc207dc56ff381b950f760cc4a4f73e13686849db4092f4279bd"
Jul 17 14:25:33.291537 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:33.291517    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"
Jul 17 14:25:33.432624 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:33.432596    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"]
Jul 17 14:25:33.434055 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:25:33.433982    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079ceb74_2221_43f4_934a_63744a10f0dd.slice/crio-c726d507a162412b52b0a1949d82268f77125f0a81099fbcee4e8cb70c88182a WatchSource:0}: Error finding container c726d507a162412b52b0a1949d82268f77125f0a81099fbcee4e8cb70c88182a: Status 404 returned error can't find the container with id c726d507a162412b52b0a1949d82268f77125f0a81099fbcee4e8cb70c88182a
Jul 17 14:25:34.304589 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:34.304503    2561 generic.go:356] "Generic (PLEG): container finished" podID="079ceb74-2221-43f4-934a-63744a10f0dd" containerID="91f3efd782e8c1239911957951f4d71d6e74032d96a4dac978d51e1b4097ddfd" exitCode=6
Jul 17 14:25:34.305055 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:34.304591    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5" event={"ID":"079ceb74-2221-43f4-934a-63744a10f0dd","Type":"ContainerDied","Data":"91f3efd782e8c1239911957951f4d71d6e74032d96a4dac978d51e1b4097ddfd"}
Jul 17 14:25:34.305055 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:34.304681    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5" event={"ID":"079ceb74-2221-43f4-934a-63744a10f0dd","Type":"ContainerStarted","Data":"c726d507a162412b52b0a1949d82268f77125f0a81099fbcee4e8cb70c88182a"}
Jul 17 14:25:34.305055 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:34.304806    2561 scope.go:117] "RemoveContainer" containerID="91f3efd782e8c1239911957951f4d71d6e74032d96a4dac978d51e1b4097ddfd"
Jul 17 14:25:34.306563 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:34.306536    2561 generic.go:356] "Generic (PLEG): container finished" podID="6768be86-6903-43a4-acce-22b9bb42eec3" containerID="f4d75b1899d0e713d250276bf82144a87ae1f420a14cfc64e27ba197642a16d8" exitCode=6
Jul 17 14:25:34.306683 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:34.306615    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt" event={"ID":"6768be86-6903-43a4-acce-22b9bb42eec3","Type":"ContainerDied","Data":"f4d75b1899d0e713d250276bf82144a87ae1f420a14cfc64e27ba197642a16d8"}
Jul 17 14:25:34.306683 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:34.306667    2561 scope.go:117] "RemoveContainer" containerID="f6ba0aae2931dc207dc56ff381b950f760cc4a4f73e13686849db4092f4279bd"
Jul 17 14:25:34.306879 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:34.306806    2561 scope.go:117] "RemoveContainer" containerID="f4d75b1899d0e713d250276bf82144a87ae1f420a14cfc64e27ba197642a16d8"
Jul 17 14:25:34.307053 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:34.307027    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-z5zkt_odh-ai-gateway-infra(6768be86-6903-43a4-acce-22b9bb42eec3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt" podUID="6768be86-6903-43a4-acce-22b9bb42eec3"
Jul 17 14:25:34.769845 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:34.769813    2561 scope.go:117] "RemoveContainer" containerID="6783ed2dc149784cb9b4860fcef759c8de7bc009c85c343f91770f4fc368e752"
Jul 17 14:25:35.314850 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:35.314752    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt" event={"ID":"ac50f1ba-3891-4f98-a5b1-e788527a19f3","Type":"ContainerStarted","Data":"30ca4a954b981ddcd8d16e7861101532d982a9e9bb14cb7024ffe71b653fcb96"}
Jul 17 14:25:35.316566 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:35.316538    2561 generic.go:356] "Generic (PLEG): container finished" podID="079ceb74-2221-43f4-934a-63744a10f0dd" containerID="3bb5dab1c60c063b40a1638f6491d8d4c64f0b12a93524008d4013626e3753bc" exitCode=6
Jul 17 14:25:35.316700 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:35.316624    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5" event={"ID":"079ceb74-2221-43f4-934a-63744a10f0dd","Type":"ContainerDied","Data":"3bb5dab1c60c063b40a1638f6491d8d4c64f0b12a93524008d4013626e3753bc"}
Jul 17 14:25:35.316700 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:35.316686    2561 scope.go:117] "RemoveContainer" containerID="91f3efd782e8c1239911957951f4d71d6e74032d96a4dac978d51e1b4097ddfd"
Jul 17 14:25:35.316825 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:35.316811    2561 scope.go:117] "RemoveContainer" containerID="3bb5dab1c60c063b40a1638f6491d8d4c64f0b12a93524008d4013626e3753bc"
Jul 17 14:25:35.317072 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:35.317040    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-77hb5_odh-ai-gateway-infra(079ceb74-2221-43f4-934a-63744a10f0dd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5" podUID="079ceb74-2221-43f4-934a-63744a10f0dd"
Jul 17 14:25:35.318567 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:35.318549    2561 scope.go:117] "RemoveContainer" containerID="f4d75b1899d0e713d250276bf82144a87ae1f420a14cfc64e27ba197642a16d8"
Jul 17 14:25:35.318726 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:35.318710    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-z5zkt_odh-ai-gateway-infra(6768be86-6903-43a4-acce-22b9bb42eec3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt" podUID="6768be86-6903-43a4-acce-22b9bb42eec3"
Jul 17 14:25:35.333492 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:35.333448    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt" podStartSLOduration=17.333434925 podStartE2EDuration="17.333434925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:25:35.33234019 +0000 UTC m=+2421.038787026" watchObservedRunningTime="2026-07-17 14:25:35.333434925 +0000 UTC m=+2421.039881760"
Jul 17 14:25:36.325982 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:36.325950    2561 generic.go:356] "Generic (PLEG): container finished" podID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerID="30ca4a954b981ddcd8d16e7861101532d982a9e9bb14cb7024ffe71b653fcb96" exitCode=7
Jul 17 14:25:36.326456 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:36.326039    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt" event={"ID":"ac50f1ba-3891-4f98-a5b1-e788527a19f3","Type":"ContainerDied","Data":"30ca4a954b981ddcd8d16e7861101532d982a9e9bb14cb7024ffe71b653fcb96"}
Jul 17 14:25:36.326456 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:36.326125    2561 scope.go:117] "RemoveContainer" containerID="6783ed2dc149784cb9b4860fcef759c8de7bc009c85c343f91770f4fc368e752"
Jul 17 14:25:36.326456 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:36.326449    2561 scope.go:117] "RemoveContainer" containerID="30ca4a954b981ddcd8d16e7861101532d982a9e9bb14cb7024ffe71b653fcb96"
Jul 17 14:25:36.326716 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:36.326694    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt_odh-ai-gateway-infra(ac50f1ba-3891-4f98-a5b1-e788527a19f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3"
Jul 17 14:25:36.327881 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:36.327863    2561 scope.go:117] "RemoveContainer" containerID="3bb5dab1c60c063b40a1638f6491d8d4c64f0b12a93524008d4013626e3753bc"
Jul 17 14:25:36.328071 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:36.328054    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-77hb5_odh-ai-gateway-infra(079ceb74-2221-43f4-934a-63744a10f0dd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5" podUID="079ceb74-2221-43f4-934a-63744a10f0dd"
Jul 17 14:25:36.342483 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:36.342460    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"]
Jul 17 14:25:37.479206 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:37.479189    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"
Jul 17 14:25:37.523707 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:37.523683    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac50f1ba-3891-4f98-a5b1-e788527a19f3-maas-api-service-ca\") pod \"ac50f1ba-3891-4f98-a5b1-e788527a19f3\" (UID: \"ac50f1ba-3891-4f98-a5b1-e788527a19f3\") "
Jul 17 14:25:37.523905 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:37.523886    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac50f1ba-3891-4f98-a5b1-e788527a19f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ac50f1ba-3891-4f98-a5b1-e788527a19f3" (UID: "ac50f1ba-3891-4f98-a5b1-e788527a19f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:25:37.524205 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:37.524174    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac50f1ba-3891-4f98-a5b1-e788527a19f3-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:38.342749 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:38.342721    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"
Jul 17 14:25:38.342943 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:38.342760    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt" event={"ID":"ac50f1ba-3891-4f98-a5b1-e788527a19f3","Type":"ContainerDied","Data":"49035af85d7830201bad25eedebedc360789c12d4a4b57f49f95b0bdfd1bc199"}
Jul 17 14:25:38.342943 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:38.342820    2561 scope.go:117] "RemoveContainer" containerID="30ca4a954b981ddcd8d16e7861101532d982a9e9bb14cb7024ffe71b653fcb96"
Jul 17 14:25:38.368617 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:38.368594    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"]
Jul 17 14:25:38.374511 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:38.374491    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-dxxgt"]
Jul 17 14:25:38.772024 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:38.771972    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" path="/var/lib/kubelet/pods/ac50f1ba-3891-4f98-a5b1-e788527a19f3/volumes"
Jul 17 14:25:46.779552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:46.779525    2561 scope.go:117] "RemoveContainer" containerID="f4d75b1899d0e713d250276bf82144a87ae1f420a14cfc64e27ba197642a16d8"
Jul 17 14:25:47.394709 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:47.394681    2561 generic.go:356] "Generic (PLEG): container finished" podID="6768be86-6903-43a4-acce-22b9bb42eec3" containerID="b1e0e4d6c21d69f80469915cfb3dae94cc9ab227bbc93826326e2135a165e276" exitCode=6
Jul 17 14:25:47.394869 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:47.394758    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt" event={"ID":"6768be86-6903-43a4-acce-22b9bb42eec3","Type":"ContainerDied","Data":"b1e0e4d6c21d69f80469915cfb3dae94cc9ab227bbc93826326e2135a165e276"}
Jul 17 14:25:47.394869 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:47.394831    2561 scope.go:117] "RemoveContainer" containerID="f4d75b1899d0e713d250276bf82144a87ae1f420a14cfc64e27ba197642a16d8"
Jul 17 14:25:47.395185 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:47.395161    2561 scope.go:117] "RemoveContainer" containerID="b1e0e4d6c21d69f80469915cfb3dae94cc9ab227bbc93826326e2135a165e276"
Jul 17 14:25:47.395437 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:47.395419    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-z5zkt_odh-ai-gateway-infra(6768be86-6903-43a4-acce-22b9bb42eec3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt" podUID="6768be86-6903-43a4-acce-22b9bb42eec3"
Jul 17 14:25:48.421117 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:48.421089    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"]
Jul 17 14:25:48.564814 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:48.564790    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"
Jul 17 14:25:48.621241 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:48.621215    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6768be86-6903-43a4-acce-22b9bb42eec3-maas-api-service-ca\") pod \"6768be86-6903-43a4-acce-22b9bb42eec3\" (UID: \"6768be86-6903-43a4-acce-22b9bb42eec3\") "
Jul 17 14:25:48.621542 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:48.621522    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6768be86-6903-43a4-acce-22b9bb42eec3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6768be86-6903-43a4-acce-22b9bb42eec3" (UID: "6768be86-6903-43a4-acce-22b9bb42eec3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:25:48.721850 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:48.721786    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6768be86-6903-43a4-acce-22b9bb42eec3-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:49.408901 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:49.408875    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"
Jul 17 14:25:49.409057 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:49.408875    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt" event={"ID":"6768be86-6903-43a4-acce-22b9bb42eec3","Type":"ContainerDied","Data":"0072017f5ed9c810e913ea8babc233ee849d3b8b76485fa92e49d5117676bc8b"}
Jul 17 14:25:49.409057 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:49.409025    2561 scope.go:117] "RemoveContainer" containerID="b1e0e4d6c21d69f80469915cfb3dae94cc9ab227bbc93826326e2135a165e276"
Jul 17 14:25:49.427348 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:49.427315    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"]
Jul 17 14:25:49.433340 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:49.433320    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-z5zkt"]
Jul 17 14:25:50.777958 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:50.777923    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6768be86-6903-43a4-acce-22b9bb42eec3" path="/var/lib/kubelet/pods/6768be86-6903-43a4-acce-22b9bb42eec3/volumes"
Jul 17 14:25:50.778606 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:50.778592    2561 scope.go:117] "RemoveContainer" containerID="3bb5dab1c60c063b40a1638f6491d8d4c64f0b12a93524008d4013626e3753bc"
Jul 17 14:25:51.426859 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:51.426824    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5" event={"ID":"079ceb74-2221-43f4-934a-63744a10f0dd","Type":"ContainerStarted","Data":"dba24166926ce7c461d774a300cdfb9761ad3c1628e2f2a625ef25f55daa28e8"}
Jul 17 14:25:51.457140 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:51.457094    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5" podStartSLOduration=19.457077084 podStartE2EDuration="19.457077084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:25:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:25:51.455115236 +0000 UTC m=+2437.161562071" watchObservedRunningTime="2026-07-17 14:25:51.457077084 +0000 UTC m=+2437.163523918"
Jul 17 14:25:52.435527 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:52.435496    2561 generic.go:356] "Generic (PLEG): container finished" podID="079ceb74-2221-43f4-934a-63744a10f0dd" containerID="dba24166926ce7c461d774a300cdfb9761ad3c1628e2f2a625ef25f55daa28e8" exitCode=6
Jul 17 14:25:52.435931 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:52.435570    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5" event={"ID":"079ceb74-2221-43f4-934a-63744a10f0dd","Type":"ContainerDied","Data":"dba24166926ce7c461d774a300cdfb9761ad3c1628e2f2a625ef25f55daa28e8"}
Jul 17 14:25:52.435931 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:52.435628    2561 scope.go:117] "RemoveContainer" containerID="3bb5dab1c60c063b40a1638f6491d8d4c64f0b12a93524008d4013626e3753bc"
Jul 17 14:25:52.436070 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:52.435995    2561 scope.go:117] "RemoveContainer" containerID="dba24166926ce7c461d774a300cdfb9761ad3c1628e2f2a625ef25f55daa28e8"
Jul 17 14:25:52.436313 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:25:52.436288    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-77hb5_odh-ai-gateway-infra(079ceb74-2221-43f4-934a-63744a10f0dd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5" podUID="079ceb74-2221-43f4-934a-63744a10f0dd"
Jul 17 14:25:52.466470 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:52.466446    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"]
Jul 17 14:25:53.596247 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:53.596220    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"
Jul 17 14:25:53.667350 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:53.667325    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/079ceb74-2221-43f4-934a-63744a10f0dd-maas-api-service-ca\") pod \"079ceb74-2221-43f4-934a-63744a10f0dd\" (UID: \"079ceb74-2221-43f4-934a-63744a10f0dd\") "
Jul 17 14:25:53.667651 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:53.667630    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/079ceb74-2221-43f4-934a-63744a10f0dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "079ceb74-2221-43f4-934a-63744a10f0dd" (UID: "079ceb74-2221-43f4-934a-63744a10f0dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:25:53.768442 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:53.768376    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/079ceb74-2221-43f4-934a-63744a10f0dd-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:25:54.450155 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:54.450128    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"
Jul 17 14:25:54.450328 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:54.450153    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5" event={"ID":"079ceb74-2221-43f4-934a-63744a10f0dd","Type":"ContainerDied","Data":"c726d507a162412b52b0a1949d82268f77125f0a81099fbcee4e8cb70c88182a"}
Jul 17 14:25:54.450328 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:54.450214    2561 scope.go:117] "RemoveContainer" containerID="dba24166926ce7c461d774a300cdfb9761ad3c1628e2f2a625ef25f55daa28e8"
Jul 17 14:25:54.474128 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:54.474105    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"]
Jul 17 14:25:54.477932 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:54.477902    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-77hb5"]
Jul 17 14:25:54.772151 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:25:54.772073    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="079ceb74-2221-43f4-934a-63744a10f0dd" path="/var/lib/kubelet/pods/079ceb74-2221-43f4-934a-63744a10f0dd/volumes"
Jul 17 14:26:08.782660 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.782629    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"]
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783032    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783043    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783052    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="079ceb74-2221-43f4-934a-63744a10f0dd" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783058    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="079ceb74-2221-43f4-934a-63744a10f0dd" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783067    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6768be86-6903-43a4-acce-22b9bb42eec3" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783073    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6768be86-6903-43a4-acce-22b9bb42eec3" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783082    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6768be86-6903-43a4-acce-22b9bb42eec3" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783087    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6768be86-6903-43a4-acce-22b9bb42eec3" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783097    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="079ceb74-2221-43f4-934a-63744a10f0dd" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783102    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="079ceb74-2221-43f4-934a-63744a10f0dd" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783109    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerName="revoke-keys"
Jul 17 14:26:08.783160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783113    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerName="revoke-keys"
Jul 17 14:26:08.783575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783240    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6768be86-6903-43a4-acce-22b9bb42eec3" containerName="revoke-keys"
Jul 17 14:26:08.783575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783272    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerName="revoke-keys"
Jul 17 14:26:08.783575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783284    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerName="revoke-keys"
Jul 17 14:26:08.783575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783295    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="079ceb74-2221-43f4-934a-63744a10f0dd" containerName="revoke-keys"
Jul 17 14:26:08.783575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783302    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="079ceb74-2221-43f4-934a-63744a10f0dd" containerName="revoke-keys"
Jul 17 14:26:08.783575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783307    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="079ceb74-2221-43f4-934a-63744a10f0dd" containerName="revoke-keys"
Jul 17 14:26:08.783575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.783314    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6768be86-6903-43a4-acce-22b9bb42eec3" containerName="revoke-keys"
Jul 17 14:26:08.787878 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.787852    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"
Jul 17 14:26:08.790854 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.790833    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:26:08.790948 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.790864    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:26:08.794886 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.794861    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"]
Jul 17 14:26:08.899733 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:08.899709    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff6cb82-0205-432c-aeba-934983fa0d39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4\" (UID: \"eff6cb82-0205-432c-aeba-934983fa0d39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"
Jul 17 14:26:09.000846 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:09.000820    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff6cb82-0205-432c-aeba-934983fa0d39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4\" (UID: \"eff6cb82-0205-432c-aeba-934983fa0d39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"
Jul 17 14:26:09.001459 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:09.001437    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff6cb82-0205-432c-aeba-934983fa0d39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4\" (UID: \"eff6cb82-0205-432c-aeba-934983fa0d39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"
Jul 17 14:26:09.099953 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:09.099896    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"
Jul 17 14:26:09.239248 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:09.239220    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"]
Jul 17 14:26:09.240904 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:26:09.240878    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff6cb82_0205_432c_aeba_934983fa0d39.slice/crio-bc692bb941f6800e8d777a5d7b5350cd7e20567a978c11c215150722e3e51e3b WatchSource:0}: Error finding container bc692bb941f6800e8d777a5d7b5350cd7e20567a978c11c215150722e3e51e3b: Status 404 returned error can't find the container with id bc692bb941f6800e8d777a5d7b5350cd7e20567a978c11c215150722e3e51e3b
Jul 17 14:26:09.533320 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:09.533292    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4" event={"ID":"eff6cb82-0205-432c-aeba-934983fa0d39","Type":"ContainerStarted","Data":"7cf51c641eab44156aa300551a6065f5835b05314dcf7e296cd41c689453a9c7"}
Jul 17 14:26:09.533454 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:09.533339    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4" event={"ID":"eff6cb82-0205-432c-aeba-934983fa0d39","Type":"ContainerStarted","Data":"bc692bb941f6800e8d777a5d7b5350cd7e20567a978c11c215150722e3e51e3b"}
Jul 17 14:26:09.552974 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:09.552927    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4" podStartSLOduration=1.5529128349999999 podStartE2EDuration="1.552912835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:26:09.551648887 +0000 UTC m=+2455.258095734" watchObservedRunningTime="2026-07-17 14:26:09.552912835 +0000 UTC m=+2455.259359669"
Jul 17 14:26:10.540560 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:10.540527    2561 generic.go:356] "Generic (PLEG): container finished" podID="eff6cb82-0205-432c-aeba-934983fa0d39" containerID="7cf51c641eab44156aa300551a6065f5835b05314dcf7e296cd41c689453a9c7" exitCode=7
Jul 17 14:26:10.541064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:10.540631    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4" event={"ID":"eff6cb82-0205-432c-aeba-934983fa0d39","Type":"ContainerDied","Data":"7cf51c641eab44156aa300551a6065f5835b05314dcf7e296cd41c689453a9c7"}
Jul 17 14:26:10.541064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:10.540838    2561 scope.go:117] "RemoveContainer" containerID="7cf51c641eab44156aa300551a6065f5835b05314dcf7e296cd41c689453a9c7"
Jul 17 14:26:11.547682 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:11.547593    2561 generic.go:356] "Generic (PLEG): container finished" podID="eff6cb82-0205-432c-aeba-934983fa0d39" containerID="ed97507b58c51db7ebcc5137f04f65e19733feab92471e5f9d51987385d66754" exitCode=7
Jul 17 14:26:11.548146 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:11.547687    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4" event={"ID":"eff6cb82-0205-432c-aeba-934983fa0d39","Type":"ContainerDied","Data":"ed97507b58c51db7ebcc5137f04f65e19733feab92471e5f9d51987385d66754"}
Jul 17 14:26:11.548146 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:11.547754    2561 scope.go:117] "RemoveContainer" containerID="7cf51c641eab44156aa300551a6065f5835b05314dcf7e296cd41c689453a9c7"
Jul 17 14:26:11.548146 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:11.547854    2561 scope.go:117] "RemoveContainer" containerID="ed97507b58c51db7ebcc5137f04f65e19733feab92471e5f9d51987385d66754"
Jul 17 14:26:11.548146 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:26:11.548053    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4_odh-ai-gateway-infra(eff6cb82-0205-432c-aeba-934983fa0d39)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4" podUID="eff6cb82-0205-432c-aeba-934983fa0d39"
Jul 17 14:26:12.555158 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:12.555128    2561 scope.go:117] "RemoveContainer" containerID="ed97507b58c51db7ebcc5137f04f65e19733feab92471e5f9d51987385d66754"
Jul 17 14:26:12.555531 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:26:12.555333    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4_odh-ai-gateway-infra(eff6cb82-0205-432c-aeba-934983fa0d39)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4" podUID="eff6cb82-0205-432c-aeba-934983fa0d39"
Jul 17 14:26:21.449988 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.449952    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"]
Jul 17 14:26:21.450522 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.450503    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6768be86-6903-43a4-acce-22b9bb42eec3" containerName="revoke-keys"
Jul 17 14:26:21.450522 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.450524    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6768be86-6903-43a4-acce-22b9bb42eec3" containerName="revoke-keys"
Jul 17 14:26:21.450647 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.450539    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="079ceb74-2221-43f4-934a-63744a10f0dd" containerName="revoke-keys"
Jul 17 14:26:21.450647 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.450545    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="079ceb74-2221-43f4-934a-63744a10f0dd" containerName="revoke-keys"
Jul 17 14:26:21.450647 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.450553    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerName="revoke-keys"
Jul 17 14:26:21.450647 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.450560    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerName="revoke-keys"
Jul 17 14:26:21.450647 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.450630    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac50f1ba-3891-4f98-a5b1-e788527a19f3" containerName="revoke-keys"
Jul 17 14:26:21.450647 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.450643    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6768be86-6903-43a4-acce-22b9bb42eec3" containerName="revoke-keys"
Jul 17 14:26:21.453700 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.453685    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"
Jul 17 14:26:21.460926 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.460869    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"]
Jul 17 14:26:21.510163 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.510127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a96d852-0114-4d00-a442-8c651a851763-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-4g5lq\" (UID: \"0a96d852-0114-4d00-a442-8c651a851763\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"
Jul 17 14:26:21.611561 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.611532    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a96d852-0114-4d00-a442-8c651a851763-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-4g5lq\" (UID: \"0a96d852-0114-4d00-a442-8c651a851763\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"
Jul 17 14:26:21.612159 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.612141    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a96d852-0114-4d00-a442-8c651a851763-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-4g5lq\" (UID: \"0a96d852-0114-4d00-a442-8c651a851763\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"
Jul 17 14:26:21.766772 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.766708    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"
Jul 17 14:26:21.905398 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:21.905372    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"]
Jul 17 14:26:21.906675 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:26:21.906650    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a96d852_0114_4d00_a442_8c651a851763.slice/crio-edaa9507753f68e55a9bfaee87b1b6f26cabf8070157c7adda624d57d8ed471a WatchSource:0}: Error finding container edaa9507753f68e55a9bfaee87b1b6f26cabf8070157c7adda624d57d8ed471a: Status 404 returned error can't find the container with id edaa9507753f68e55a9bfaee87b1b6f26cabf8070157c7adda624d57d8ed471a
Jul 17 14:26:22.608899 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:22.608857    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq" event={"ID":"0a96d852-0114-4d00-a442-8c651a851763","Type":"ContainerStarted","Data":"7ed4e9af25e7ba164079a0016254206e898d92cdc8f5233f2bbc358ee19a1b90"}
Jul 17 14:26:22.609324 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:22.608927    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq" event={"ID":"0a96d852-0114-4d00-a442-8c651a851763","Type":"ContainerStarted","Data":"edaa9507753f68e55a9bfaee87b1b6f26cabf8070157c7adda624d57d8ed471a"}
Jul 17 14:26:22.625080 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:22.625026    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq" podStartSLOduration=1.6249831000000001 podStartE2EDuration="1.6249831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:26:22.624243549 +0000 UTC m=+2468.330690384" watchObservedRunningTime="2026-07-17 14:26:22.6249831 +0000 UTC m=+2468.331429935"
Jul 17 14:26:23.615763 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:23.615731    2561 generic.go:356] "Generic (PLEG): container finished" podID="0a96d852-0114-4d00-a442-8c651a851763" containerID="7ed4e9af25e7ba164079a0016254206e898d92cdc8f5233f2bbc358ee19a1b90" exitCode=6
Jul 17 14:26:23.616190 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:23.615833    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq" event={"ID":"0a96d852-0114-4d00-a442-8c651a851763","Type":"ContainerDied","Data":"7ed4e9af25e7ba164079a0016254206e898d92cdc8f5233f2bbc358ee19a1b90"}
Jul 17 14:26:23.616190 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:23.616036    2561 scope.go:117] "RemoveContainer" containerID="7ed4e9af25e7ba164079a0016254206e898d92cdc8f5233f2bbc358ee19a1b90"
Jul 17 14:26:24.629495 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:24.629456    2561 generic.go:356] "Generic (PLEG): container finished" podID="0a96d852-0114-4d00-a442-8c651a851763" containerID="8306faada3405a0d9bfe6754966bb5e56fb35e3f846bbfca54550c093fe8b1a9" exitCode=6
Jul 17 14:26:24.629973 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:24.629498    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq" event={"ID":"0a96d852-0114-4d00-a442-8c651a851763","Type":"ContainerDied","Data":"8306faada3405a0d9bfe6754966bb5e56fb35e3f846bbfca54550c093fe8b1a9"}
Jul 17 14:26:24.629973 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:24.629600    2561 scope.go:117] "RemoveContainer" containerID="7ed4e9af25e7ba164079a0016254206e898d92cdc8f5233f2bbc358ee19a1b90"
Jul 17 14:26:24.629973 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:24.629806    2561 scope.go:117] "RemoveContainer" containerID="8306faada3405a0d9bfe6754966bb5e56fb35e3f846bbfca54550c093fe8b1a9"
Jul 17 14:26:24.630140 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:26:24.630072    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-4g5lq_odh-ai-gateway-infra(0a96d852-0114-4d00-a442-8c651a851763)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq" podUID="0a96d852-0114-4d00-a442-8c651a851763"
Jul 17 14:26:25.636265 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:25.636239    2561 scope.go:117] "RemoveContainer" containerID="8306faada3405a0d9bfe6754966bb5e56fb35e3f846bbfca54550c093fe8b1a9"
Jul 17 14:26:25.636708 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:26:25.636477    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-4g5lq_odh-ai-gateway-infra(0a96d852-0114-4d00-a442-8c651a851763)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq" podUID="0a96d852-0114-4d00-a442-8c651a851763"
Jul 17 14:26:27.766088 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:27.766053    2561 scope.go:117] "RemoveContainer" containerID="ed97507b58c51db7ebcc5137f04f65e19733feab92471e5f9d51987385d66754"
Jul 17 14:26:28.652732 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:28.652655    2561 generic.go:356] "Generic (PLEG): container finished" podID="eff6cb82-0205-432c-aeba-934983fa0d39" containerID="516014c88a55cabab0024a4cb53a42307c35f25ea056d737d6650493934a7ed0" exitCode=7
Jul 17 14:26:28.652891 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:28.652735    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4" event={"ID":"eff6cb82-0205-432c-aeba-934983fa0d39","Type":"ContainerDied","Data":"516014c88a55cabab0024a4cb53a42307c35f25ea056d737d6650493934a7ed0"}
Jul 17 14:26:28.652891 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:28.652805    2561 scope.go:117] "RemoveContainer" containerID="ed97507b58c51db7ebcc5137f04f65e19733feab92471e5f9d51987385d66754"
Jul 17 14:26:28.653151 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:28.653137    2561 scope.go:117] "RemoveContainer" containerID="516014c88a55cabab0024a4cb53a42307c35f25ea056d737d6650493934a7ed0"
Jul 17 14:26:28.653373 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:26:28.653354    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4_odh-ai-gateway-infra(eff6cb82-0205-432c-aeba-934983fa0d39)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4" podUID="eff6cb82-0205-432c-aeba-934983fa0d39"
Jul 17 14:26:29.681923 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:29.681890    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"]
Jul 17 14:26:29.824885 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:29.824861    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"
Jul 17 14:26:29.886237 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:29.886212    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff6cb82-0205-432c-aeba-934983fa0d39-maas-api-service-ca\") pod \"eff6cb82-0205-432c-aeba-934983fa0d39\" (UID: \"eff6cb82-0205-432c-aeba-934983fa0d39\") "
Jul 17 14:26:29.886563 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:29.886539    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eff6cb82-0205-432c-aeba-934983fa0d39-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eff6cb82-0205-432c-aeba-934983fa0d39" (UID: "eff6cb82-0205-432c-aeba-934983fa0d39"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:26:29.987223 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:29.987158    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff6cb82-0205-432c-aeba-934983fa0d39-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:26:30.665990 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:30.665953    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"
Jul 17 14:26:30.666209 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:30.665988    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4" event={"ID":"eff6cb82-0205-432c-aeba-934983fa0d39","Type":"ContainerDied","Data":"bc692bb941f6800e8d777a5d7b5350cd7e20567a978c11c215150722e3e51e3b"}
Jul 17 14:26:30.666209 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:30.666078    2561 scope.go:117] "RemoveContainer" containerID="516014c88a55cabab0024a4cb53a42307c35f25ea056d737d6650493934a7ed0"
Jul 17 14:26:30.688770 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:30.688749    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"]
Jul 17 14:26:30.697843 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:30.697821    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-kfnk4"]
Jul 17 14:26:30.772438 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:30.772408    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff6cb82-0205-432c-aeba-934983fa0d39" path="/var/lib/kubelet/pods/eff6cb82-0205-432c-aeba-934983fa0d39/volumes"
Jul 17 14:26:32.152362 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.152324    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"]
Jul 17 14:26:32.152796 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.152782    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff6cb82-0205-432c-aeba-934983fa0d39" containerName="revoke-keys"
Jul 17 14:26:32.152842 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.152798    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff6cb82-0205-432c-aeba-934983fa0d39" containerName="revoke-keys"
Jul 17 14:26:32.152842 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.152827    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff6cb82-0205-432c-aeba-934983fa0d39" containerName="revoke-keys"
Jul 17 14:26:32.152842 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.152833    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff6cb82-0205-432c-aeba-934983fa0d39" containerName="revoke-keys"
Jul 17 14:26:32.152944 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.152908    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff6cb82-0205-432c-aeba-934983fa0d39" containerName="revoke-keys"
Jul 17 14:26:32.152944 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.152916    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff6cb82-0205-432c-aeba-934983fa0d39" containerName="revoke-keys"
Jul 17 14:26:32.157460 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.157445    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"
Jul 17 14:26:32.162507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.162478    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"]
Jul 17 14:26:32.206563 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.206534    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ccf9b5a-a720-4929-9578-fd7200905999-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm\" (UID: \"1ccf9b5a-a720-4929-9578-fd7200905999\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"
Jul 17 14:26:32.307937 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.307911    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ccf9b5a-a720-4929-9578-fd7200905999-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm\" (UID: \"1ccf9b5a-a720-4929-9578-fd7200905999\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"
Jul 17 14:26:32.308499 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.308483    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ccf9b5a-a720-4929-9578-fd7200905999-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm\" (UID: \"1ccf9b5a-a720-4929-9578-fd7200905999\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"
Jul 17 14:26:32.470300 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.470278    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"
Jul 17 14:26:32.816388 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:32.816363    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"]
Jul 17 14:26:32.818127 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:26:32.818098    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ccf9b5a_a720_4929_9578_fd7200905999.slice/crio-f5711ba432573bd52be794da16853e46ede8d593a58927d6ff69ba961f2682a3 WatchSource:0}: Error finding container f5711ba432573bd52be794da16853e46ede8d593a58927d6ff69ba961f2682a3: Status 404 returned error can't find the container with id f5711ba432573bd52be794da16853e46ede8d593a58927d6ff69ba961f2682a3
Jul 17 14:26:33.684403 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:33.684373    2561 generic.go:356] "Generic (PLEG): container finished" podID="1ccf9b5a-a720-4929-9578-fd7200905999" containerID="038549e12fe7a53a889debf2b8ebef7bf481f4bc1112b22b825eb43ccb53956a" exitCode=6
Jul 17 14:26:33.684758 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:33.684420    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm" event={"ID":"1ccf9b5a-a720-4929-9578-fd7200905999","Type":"ContainerDied","Data":"038549e12fe7a53a889debf2b8ebef7bf481f4bc1112b22b825eb43ccb53956a"}
Jul 17 14:26:33.684758 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:33.684455    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm" event={"ID":"1ccf9b5a-a720-4929-9578-fd7200905999","Type":"ContainerStarted","Data":"f5711ba432573bd52be794da16853e46ede8d593a58927d6ff69ba961f2682a3"}
Jul 17 14:26:33.684758 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:33.684621    2561 scope.go:117] "RemoveContainer" containerID="038549e12fe7a53a889debf2b8ebef7bf481f4bc1112b22b825eb43ccb53956a"
Jul 17 14:26:34.692422 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:34.692342    2561 generic.go:356] "Generic (PLEG): container finished" podID="1ccf9b5a-a720-4929-9578-fd7200905999" containerID="ffd1cd2f5c2c52c60291a2bf26dd256fb07644040340f2b3f46c1c9d6982d840" exitCode=6
Jul 17 14:26:34.692422 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:34.692385    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm" event={"ID":"1ccf9b5a-a720-4929-9578-fd7200905999","Type":"ContainerDied","Data":"ffd1cd2f5c2c52c60291a2bf26dd256fb07644040340f2b3f46c1c9d6982d840"}
Jul 17 14:26:34.692842 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:34.692466    2561 scope.go:117] "RemoveContainer" containerID="038549e12fe7a53a889debf2b8ebef7bf481f4bc1112b22b825eb43ccb53956a"
Jul 17 14:26:34.692842 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:34.692663    2561 scope.go:117] "RemoveContainer" containerID="ffd1cd2f5c2c52c60291a2bf26dd256fb07644040340f2b3f46c1c9d6982d840"
Jul 17 14:26:34.692912 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:26:34.692879    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm_odh-ai-gateway-infra(1ccf9b5a-a720-4929-9578-fd7200905999)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm" podUID="1ccf9b5a-a720-4929-9578-fd7200905999"
Jul 17 14:26:35.699694 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:35.699671    2561 scope.go:117] "RemoveContainer" containerID="ffd1cd2f5c2c52c60291a2bf26dd256fb07644040340f2b3f46c1c9d6982d840"
Jul 17 14:26:35.700109 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:26:35.699844    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm_odh-ai-gateway-infra(1ccf9b5a-a720-4929-9578-fd7200905999)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm" podUID="1ccf9b5a-a720-4929-9578-fd7200905999"
Jul 17 14:26:37.765929 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:37.765900    2561 scope.go:117] "RemoveContainer" containerID="8306faada3405a0d9bfe6754966bb5e56fb35e3f846bbfca54550c093fe8b1a9"
Jul 17 14:26:38.718540 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:38.718512    2561 generic.go:356] "Generic (PLEG): container finished" podID="0a96d852-0114-4d00-a442-8c651a851763" containerID="93d179daafe37a3c550176ee3160f8edb21108cde88eb5905f91cb802f83c1bf" exitCode=6
Jul 17 14:26:38.718732 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:38.718595    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq" event={"ID":"0a96d852-0114-4d00-a442-8c651a851763","Type":"ContainerDied","Data":"93d179daafe37a3c550176ee3160f8edb21108cde88eb5905f91cb802f83c1bf"}
Jul 17 14:26:38.718732 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:38.718671    2561 scope.go:117] "RemoveContainer" containerID="8306faada3405a0d9bfe6754966bb5e56fb35e3f846bbfca54550c093fe8b1a9"
Jul 17 14:26:38.719024 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:38.718999    2561 scope.go:117] "RemoveContainer" containerID="93d179daafe37a3c550176ee3160f8edb21108cde88eb5905f91cb802f83c1bf"
Jul 17 14:26:38.719269 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:26:38.719241    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-4g5lq_odh-ai-gateway-infra(0a96d852-0114-4d00-a442-8c651a851763)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq" podUID="0a96d852-0114-4d00-a442-8c651a851763"
Jul 17 14:26:39.746540 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:39.746502    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"]
Jul 17 14:26:39.911511 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:39.911485    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"
Jul 17 14:26:39.976723 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:39.976689    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a96d852-0114-4d00-a442-8c651a851763-maas-api-service-ca\") pod \"0a96d852-0114-4d00-a442-8c651a851763\" (UID: \"0a96d852-0114-4d00-a442-8c651a851763\") "
Jul 17 14:26:39.977086 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:39.977062    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a96d852-0114-4d00-a442-8c651a851763-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a96d852-0114-4d00-a442-8c651a851763" (UID: "0a96d852-0114-4d00-a442-8c651a851763"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:26:40.078555 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:40.078475    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a96d852-0114-4d00-a442-8c651a851763-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:26:40.732425 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:40.732396    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"
Jul 17 14:26:40.732622 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:40.732437    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq" event={"ID":"0a96d852-0114-4d00-a442-8c651a851763","Type":"ContainerDied","Data":"edaa9507753f68e55a9bfaee87b1b6f26cabf8070157c7adda624d57d8ed471a"}
Jul 17 14:26:40.732622 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:40.732497    2561 scope.go:117] "RemoveContainer" containerID="93d179daafe37a3c550176ee3160f8edb21108cde88eb5905f91cb802f83c1bf"
Jul 17 14:26:40.756543 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:40.756511    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"]
Jul 17 14:26:40.764613 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:40.764587    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-4g5lq"]
Jul 17 14:26:40.771812 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:40.771787    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a96d852-0114-4d00-a442-8c651a851763" path="/var/lib/kubelet/pods/0a96d852-0114-4d00-a442-8c651a851763/volumes"
Jul 17 14:26:47.765905 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:47.765874    2561 scope.go:117] "RemoveContainer" containerID="ffd1cd2f5c2c52c60291a2bf26dd256fb07644040340f2b3f46c1c9d6982d840"
Jul 17 14:26:48.777707 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:48.777676    2561 generic.go:356] "Generic (PLEG): container finished" podID="1ccf9b5a-a720-4929-9578-fd7200905999" containerID="e6d7e90ee2bf7bc4d14a75d499fcf858f65ae0ef3fb86f80febd1ad80e29da84" exitCode=6
Jul 17 14:26:48.778176 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:48.777718    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm" event={"ID":"1ccf9b5a-a720-4929-9578-fd7200905999","Type":"ContainerDied","Data":"e6d7e90ee2bf7bc4d14a75d499fcf858f65ae0ef3fb86f80febd1ad80e29da84"}
Jul 17 14:26:48.778176 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:48.777765    2561 scope.go:117] "RemoveContainer" containerID="ffd1cd2f5c2c52c60291a2bf26dd256fb07644040340f2b3f46c1c9d6982d840"
Jul 17 14:26:48.778176 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:48.778135    2561 scope.go:117] "RemoveContainer" containerID="e6d7e90ee2bf7bc4d14a75d499fcf858f65ae0ef3fb86f80febd1ad80e29da84"
Jul 17 14:26:48.778453 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:26:48.778428    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm_odh-ai-gateway-infra(1ccf9b5a-a720-4929-9578-fd7200905999)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm" podUID="1ccf9b5a-a720-4929-9578-fd7200905999"
Jul 17 14:26:49.804470 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:49.804421    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"]
Jul 17 14:26:49.947726 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:49.947702    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"
Jul 17 14:26:50.078229 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:50.078139    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ccf9b5a-a720-4929-9578-fd7200905999-maas-api-service-ca\") pod \"1ccf9b5a-a720-4929-9578-fd7200905999\" (UID: \"1ccf9b5a-a720-4929-9578-fd7200905999\") "
Jul 17 14:26:50.078509 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:50.078487    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ccf9b5a-a720-4929-9578-fd7200905999-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1ccf9b5a-a720-4929-9578-fd7200905999" (UID: "1ccf9b5a-a720-4929-9578-fd7200905999"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:26:50.179535 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:50.179500    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ccf9b5a-a720-4929-9578-fd7200905999-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:26:50.791147 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:50.791124    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"
Jul 17 14:26:50.791342 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:50.791118    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm" event={"ID":"1ccf9b5a-a720-4929-9578-fd7200905999","Type":"ContainerDied","Data":"f5711ba432573bd52be794da16853e46ede8d593a58927d6ff69ba961f2682a3"}
Jul 17 14:26:50.791342 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:50.791254    2561 scope.go:117] "RemoveContainer" containerID="e6d7e90ee2bf7bc4d14a75d499fcf858f65ae0ef3fb86f80febd1ad80e29da84"
Jul 17 14:26:50.808946 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:50.808920    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"]
Jul 17 14:26:50.818732 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:50.818703    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-rw8nm"]
Jul 17 14:26:52.772608 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:26:52.772572    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ccf9b5a-a720-4929-9578-fd7200905999" path="/var/lib/kubelet/pods/1ccf9b5a-a720-4929-9578-fd7200905999/volumes"
Jul 17 14:27:33.072060 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072022    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"]
Jul 17 14:27:33.072482 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072436    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a96d852-0114-4d00-a442-8c651a851763" containerName="revoke-keys"
Jul 17 14:27:33.072482 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072447    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a96d852-0114-4d00-a442-8c651a851763" containerName="revoke-keys"
Jul 17 14:27:33.072482 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072458    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff6cb82-0205-432c-aeba-934983fa0d39" containerName="revoke-keys"
Jul 17 14:27:33.072482 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072464    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff6cb82-0205-432c-aeba-934983fa0d39" containerName="revoke-keys"
Jul 17 14:27:33.072482 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072472    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ccf9b5a-a720-4929-9578-fd7200905999" containerName="revoke-keys"
Jul 17 14:27:33.072482 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072479    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ccf9b5a-a720-4929-9578-fd7200905999" containerName="revoke-keys"
Jul 17 14:27:33.072675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072492    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a96d852-0114-4d00-a442-8c651a851763" containerName="revoke-keys"
Jul 17 14:27:33.072675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072497    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a96d852-0114-4d00-a442-8c651a851763" containerName="revoke-keys"
Jul 17 14:27:33.072675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072508    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ccf9b5a-a720-4929-9578-fd7200905999" containerName="revoke-keys"
Jul 17 14:27:33.072675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072513    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ccf9b5a-a720-4929-9578-fd7200905999" containerName="revoke-keys"
Jul 17 14:27:33.072675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072573    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a96d852-0114-4d00-a442-8c651a851763" containerName="revoke-keys"
Jul 17 14:27:33.072675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072581    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ccf9b5a-a720-4929-9578-fd7200905999" containerName="revoke-keys"
Jul 17 14:27:33.072675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072589    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ccf9b5a-a720-4929-9578-fd7200905999" containerName="revoke-keys"
Jul 17 14:27:33.072675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072594    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff6cb82-0205-432c-aeba-934983fa0d39" containerName="revoke-keys"
Jul 17 14:27:33.072675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.072600    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a96d852-0114-4d00-a442-8c651a851763" containerName="revoke-keys"
Jul 17 14:27:33.075568 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.075552    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"
Jul 17 14:27:33.078416 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.078392    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:27:33.078416 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.078406    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:27:33.084585 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.084564    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"]
Jul 17 14:27:33.173018 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.172988    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057ae03b-a84c-4ae8-8db0-485facfe6a89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2\" (UID: \"057ae03b-a84c-4ae8-8db0-485facfe6a89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"
Jul 17 14:27:33.274554 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.274528    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057ae03b-a84c-4ae8-8db0-485facfe6a89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2\" (UID: \"057ae03b-a84c-4ae8-8db0-485facfe6a89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"
Jul 17 14:27:33.275111 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.275096    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057ae03b-a84c-4ae8-8db0-485facfe6a89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2\" (UID: \"057ae03b-a84c-4ae8-8db0-485facfe6a89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"
Jul 17 14:27:33.387043 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.386958    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"
Jul 17 14:27:33.747046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:33.747020    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"]
Jul 17 14:27:33.749077 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:27:33.749049    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057ae03b_a84c_4ae8_8db0_485facfe6a89.slice/crio-9badf70dfe8cfc79e3d0c2633c60a27504b4ae4eda8db677222015829c2e0b63 WatchSource:0}: Error finding container 9badf70dfe8cfc79e3d0c2633c60a27504b4ae4eda8db677222015829c2e0b63: Status 404 returned error can't find the container with id 9badf70dfe8cfc79e3d0c2633c60a27504b4ae4eda8db677222015829c2e0b63
Jul 17 14:27:34.017455 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:34.017392    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2" event={"ID":"057ae03b-a84c-4ae8-8db0-485facfe6a89","Type":"ContainerStarted","Data":"9badf70dfe8cfc79e3d0c2633c60a27504b4ae4eda8db677222015829c2e0b63"}
Jul 17 14:27:35.024333 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:35.024293    2561 generic.go:356] "Generic (PLEG): container finished" podID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerID="3c98e91baf9c377323bdbad6dc8b1b00a95756673013851354759de8d9b68613" exitCode=6
Jul 17 14:27:35.024684 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:35.024375    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2" event={"ID":"057ae03b-a84c-4ae8-8db0-485facfe6a89","Type":"ContainerDied","Data":"3c98e91baf9c377323bdbad6dc8b1b00a95756673013851354759de8d9b68613"}
Jul 17 14:27:35.024684 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:35.024531    2561 scope.go:117] "RemoveContainer" containerID="3c98e91baf9c377323bdbad6dc8b1b00a95756673013851354759de8d9b68613"
Jul 17 14:27:36.032646 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:36.032609    2561 generic.go:356] "Generic (PLEG): container finished" podID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerID="b61e7d286f38ae01859120b6b0b748b43c5ceb49c7ad9720b4c814fbd3293734" exitCode=6
Jul 17 14:27:36.033080 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:36.032656    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2" event={"ID":"057ae03b-a84c-4ae8-8db0-485facfe6a89","Type":"ContainerDied","Data":"b61e7d286f38ae01859120b6b0b748b43c5ceb49c7ad9720b4c814fbd3293734"}
Jul 17 14:27:36.033080 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:36.032741    2561 scope.go:117] "RemoveContainer" containerID="3c98e91baf9c377323bdbad6dc8b1b00a95756673013851354759de8d9b68613"
Jul 17 14:27:36.033080 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:36.032914    2561 scope.go:117] "RemoveContainer" containerID="b61e7d286f38ae01859120b6b0b748b43c5ceb49c7ad9720b4c814fbd3293734"
Jul 17 14:27:36.033279 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:27:36.033195    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2_odh-ai-gateway-infra(057ae03b-a84c-4ae8-8db0-485facfe6a89)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89"
Jul 17 14:27:37.039637 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:37.039604    2561 scope.go:117] "RemoveContainer" containerID="b61e7d286f38ae01859120b6b0b748b43c5ceb49c7ad9720b4c814fbd3293734"
Jul 17 14:27:37.040136 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:27:37.039843    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2_odh-ai-gateway-infra(057ae03b-a84c-4ae8-8db0-485facfe6a89)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89"
Jul 17 14:27:51.766128 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:51.766089    2561 scope.go:117] "RemoveContainer" containerID="b61e7d286f38ae01859120b6b0b748b43c5ceb49c7ad9720b4c814fbd3293734"
Jul 17 14:27:52.119611 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:52.119534    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2" event={"ID":"057ae03b-a84c-4ae8-8db0-485facfe6a89","Type":"ContainerStarted","Data":"cab078f705f79a6c1d3a6dddcfe6ca1efb26dae1f9abbf9d9393699884ba5151"}
Jul 17 14:27:52.137717 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:52.137669    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2" podStartSLOduration=19.137655287 podStartE2EDuration="19.137655287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:27:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:27:52.136513043 +0000 UTC m=+2557.842959877" watchObservedRunningTime="2026-07-17 14:27:52.137655287 +0000 UTC m=+2557.844102121"
Jul 17 14:27:53.126585 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:53.126550    2561 generic.go:356] "Generic (PLEG): container finished" podID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerID="cab078f705f79a6c1d3a6dddcfe6ca1efb26dae1f9abbf9d9393699884ba5151" exitCode=6
Jul 17 14:27:53.127032 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:53.126622    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2" event={"ID":"057ae03b-a84c-4ae8-8db0-485facfe6a89","Type":"ContainerDied","Data":"cab078f705f79a6c1d3a6dddcfe6ca1efb26dae1f9abbf9d9393699884ba5151"}
Jul 17 14:27:53.127032 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:53.126679    2561 scope.go:117] "RemoveContainer" containerID="b61e7d286f38ae01859120b6b0b748b43c5ceb49c7ad9720b4c814fbd3293734"
Jul 17 14:27:53.127032 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:53.126920    2561 scope.go:117] "RemoveContainer" containerID="cab078f705f79a6c1d3a6dddcfe6ca1efb26dae1f9abbf9d9393699884ba5151"
Jul 17 14:27:53.127238 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:27:53.127208    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2_odh-ai-gateway-infra(057ae03b-a84c-4ae8-8db0-485facfe6a89)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89"
Jul 17 14:27:53.163453 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:53.163427    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"]
Jul 17 14:27:54.272455 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:54.272433    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"
Jul 17 14:27:54.361746 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:54.361717    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057ae03b-a84c-4ae8-8db0-485facfe6a89-maas-api-service-ca\") pod \"057ae03b-a84c-4ae8-8db0-485facfe6a89\" (UID: \"057ae03b-a84c-4ae8-8db0-485facfe6a89\") "
Jul 17 14:27:54.362050 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:54.362029    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/057ae03b-a84c-4ae8-8db0-485facfe6a89-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "057ae03b-a84c-4ae8-8db0-485facfe6a89" (UID: "057ae03b-a84c-4ae8-8db0-485facfe6a89"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:27:54.462580 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:54.462557    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057ae03b-a84c-4ae8-8db0-485facfe6a89-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:27:55.141029 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:55.140942    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"
Jul 17 14:27:55.141029 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:55.140974    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2" event={"ID":"057ae03b-a84c-4ae8-8db0-485facfe6a89","Type":"ContainerDied","Data":"9badf70dfe8cfc79e3d0c2633c60a27504b4ae4eda8db677222015829c2e0b63"}
Jul 17 14:27:55.141249 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:55.141044    2561 scope.go:117] "RemoveContainer" containerID="cab078f705f79a6c1d3a6dddcfe6ca1efb26dae1f9abbf9d9393699884ba5151"
Jul 17 14:27:55.164154 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:55.164128    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"]
Jul 17 14:27:55.166229 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:55.166208    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-988d1f69-fp9x2"]
Jul 17 14:27:56.772887 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:56.772851    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89" path="/var/lib/kubelet/pods/057ae03b-a84c-4ae8-8db0-485facfe6a89/volumes"
Jul 17 14:27:58.897278 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897246    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"]
Jul 17 14:27:58.897809 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897791    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerName="revoke-keys"
Jul 17 14:27:58.897855 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897811    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerName="revoke-keys"
Jul 17 14:27:58.897855 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897822    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerName="revoke-keys"
Jul 17 14:27:58.897855 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897828    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerName="revoke-keys"
Jul 17 14:27:58.897855 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897841    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerName="revoke-keys"
Jul 17 14:27:58.897855 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897846    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerName="revoke-keys"
Jul 17 14:27:58.898046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897858    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ccf9b5a-a720-4929-9578-fd7200905999" containerName="revoke-keys"
Jul 17 14:27:58.898046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897863    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ccf9b5a-a720-4929-9578-fd7200905999" containerName="revoke-keys"
Jul 17 14:27:58.898046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897875    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a96d852-0114-4d00-a442-8c651a851763" containerName="revoke-keys"
Jul 17 14:27:58.898046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897880    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a96d852-0114-4d00-a442-8c651a851763" containerName="revoke-keys"
Jul 17 14:27:58.898046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897945    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ccf9b5a-a720-4929-9578-fd7200905999" containerName="revoke-keys"
Jul 17 14:27:58.898046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897954    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerName="revoke-keys"
Jul 17 14:27:58.898046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897962    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerName="revoke-keys"
Jul 17 14:27:58.898046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.897970    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a96d852-0114-4d00-a442-8c651a851763" containerName="revoke-keys"
Jul 17 14:27:58.902529 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.902512    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"
Jul 17 14:27:58.905120 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.905096    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:27:58.905236 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.905150    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:27:58.912257 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.912226    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"]
Jul 17 14:27:58.998772 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:58.998747    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/356a5611-a29d-4a64-a840-e3ee09af90cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b\" (UID: \"356a5611-a29d-4a64-a840-e3ee09af90cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"
Jul 17 14:27:59.099946 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:59.099913    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/356a5611-a29d-4a64-a840-e3ee09af90cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b\" (UID: \"356a5611-a29d-4a64-a840-e3ee09af90cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"
Jul 17 14:27:59.100534 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:59.100517    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/356a5611-a29d-4a64-a840-e3ee09af90cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b\" (UID: \"356a5611-a29d-4a64-a840-e3ee09af90cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"
Jul 17 14:27:59.214539 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:59.214515    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"
Jul 17 14:27:59.353949 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:27:59.353922    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"]
Jul 17 14:27:59.355923 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:27:59.355898    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod356a5611_a29d_4a64_a840_e3ee09af90cf.slice/crio-d58b3052baeccf7744051db58b3557a8ce324669933771bcc0c7de4a63dae4a0 WatchSource:0}: Error finding container d58b3052baeccf7744051db58b3557a8ce324669933771bcc0c7de4a63dae4a0: Status 404 returned error can't find the container with id d58b3052baeccf7744051db58b3557a8ce324669933771bcc0c7de4a63dae4a0
Jul 17 14:28:00.169923 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:00.169889    2561 generic.go:356] "Generic (PLEG): container finished" podID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerID="89231c8b6330913ee25f22215d6e0ff86b2f34cb002b26579b7373b403851df0" exitCode=7
Jul 17 14:28:00.170418 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:00.169980    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b" event={"ID":"356a5611-a29d-4a64-a840-e3ee09af90cf","Type":"ContainerDied","Data":"89231c8b6330913ee25f22215d6e0ff86b2f34cb002b26579b7373b403851df0"}
Jul 17 14:28:00.170418 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:00.170045    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b" event={"ID":"356a5611-a29d-4a64-a840-e3ee09af90cf","Type":"ContainerStarted","Data":"d58b3052baeccf7744051db58b3557a8ce324669933771bcc0c7de4a63dae4a0"}
Jul 17 14:28:00.170418 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:00.170216    2561 scope.go:117] "RemoveContainer" containerID="89231c8b6330913ee25f22215d6e0ff86b2f34cb002b26579b7373b403851df0"
Jul 17 14:28:01.177464 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:01.177434    2561 generic.go:356] "Generic (PLEG): container finished" podID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerID="ceceb7d7bed8299178e4c2434ae5a2654501f26c40f284e3010333745bf8c8e9" exitCode=7
Jul 17 14:28:01.177897 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:01.177512    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b" event={"ID":"356a5611-a29d-4a64-a840-e3ee09af90cf","Type":"ContainerDied","Data":"ceceb7d7bed8299178e4c2434ae5a2654501f26c40f284e3010333745bf8c8e9"}
Jul 17 14:28:01.177897 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:01.177583    2561 scope.go:117] "RemoveContainer" containerID="89231c8b6330913ee25f22215d6e0ff86b2f34cb002b26579b7373b403851df0"
Jul 17 14:28:01.177897 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:01.177769    2561 scope.go:117] "RemoveContainer" containerID="ceceb7d7bed8299178e4c2434ae5a2654501f26c40f284e3010333745bf8c8e9"
Jul 17 14:28:01.178117 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:28:01.178032    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b_odh-ai-gateway-infra(356a5611-a29d-4a64-a840-e3ee09af90cf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf"
Jul 17 14:28:02.185026 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:02.184977    2561 scope.go:117] "RemoveContainer" containerID="ceceb7d7bed8299178e4c2434ae5a2654501f26c40f284e3010333745bf8c8e9"
Jul 17 14:28:02.185427 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:28:02.185213    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b_odh-ai-gateway-infra(356a5611-a29d-4a64-a840-e3ee09af90cf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf"
Jul 17 14:28:12.246217 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:12.246176    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"]
Jul 17 14:28:12.246863 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:12.246841    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="057ae03b-a84c-4ae8-8db0-485facfe6a89" containerName="revoke-keys"
Jul 17 14:28:12.251223 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:12.251201    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"
Jul 17 14:28:12.262818 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:12.262794    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"]
Jul 17 14:28:12.315920 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:12.315892    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37f6bc81-828b-4992-9273-2abd3c9ef683-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8\" (UID: \"37f6bc81-828b-4992-9273-2abd3c9ef683\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"
Jul 17 14:28:12.416474 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:12.416447    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37f6bc81-828b-4992-9273-2abd3c9ef683-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8\" (UID: \"37f6bc81-828b-4992-9273-2abd3c9ef683\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"
Jul 17 14:28:12.417000 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:12.416983    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37f6bc81-828b-4992-9273-2abd3c9ef683-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8\" (UID: \"37f6bc81-828b-4992-9273-2abd3c9ef683\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"
Jul 17 14:28:12.563333 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:12.563270    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"
Jul 17 14:28:12.706630 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:12.706606    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"]
Jul 17 14:28:12.709392 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:28:12.709363    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f6bc81_828b_4992_9273_2abd3c9ef683.slice/crio-68838ffc5ca6b0ab0d4e812c5e5a04b7dca71c37c19a250a461b82116ad0d128 WatchSource:0}: Error finding container 68838ffc5ca6b0ab0d4e812c5e5a04b7dca71c37c19a250a461b82116ad0d128: Status 404 returned error can't find the container with id 68838ffc5ca6b0ab0d4e812c5e5a04b7dca71c37c19a250a461b82116ad0d128
Jul 17 14:28:13.242970 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:13.242884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8" event={"ID":"37f6bc81-828b-4992-9273-2abd3c9ef683","Type":"ContainerStarted","Data":"66e59f8db0aa8c454a2ca2bc04d0b8dc1900faa6dbfc3d6696ec16a6309da9c2"}
Jul 17 14:28:13.242970 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:13.242937    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8" event={"ID":"37f6bc81-828b-4992-9273-2abd3c9ef683","Type":"ContainerStarted","Data":"68838ffc5ca6b0ab0d4e812c5e5a04b7dca71c37c19a250a461b82116ad0d128"}
Jul 17 14:28:13.263322 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:13.263271    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8" podStartSLOduration=1.263257001 podStartE2EDuration="1.263257001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:28:13.260781963 +0000 UTC m=+2578.967228811" watchObservedRunningTime="2026-07-17 14:28:13.263257001 +0000 UTC m=+2578.969703834"
Jul 17 14:28:14.250423 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:14.250383    2561 generic.go:356] "Generic (PLEG): container finished" podID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerID="66e59f8db0aa8c454a2ca2bc04d0b8dc1900faa6dbfc3d6696ec16a6309da9c2" exitCode=6
Jul 17 14:28:14.250587 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:14.250465    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8" event={"ID":"37f6bc81-828b-4992-9273-2abd3c9ef683","Type":"ContainerDied","Data":"66e59f8db0aa8c454a2ca2bc04d0b8dc1900faa6dbfc3d6696ec16a6309da9c2"}
Jul 17 14:28:14.250671 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:14.250657    2561 scope.go:117] "RemoveContainer" containerID="66e59f8db0aa8c454a2ca2bc04d0b8dc1900faa6dbfc3d6696ec16a6309da9c2"
Jul 17 14:28:14.772034 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:14.771940    2561 scope.go:117] "RemoveContainer" containerID="ceceb7d7bed8299178e4c2434ae5a2654501f26c40f284e3010333745bf8c8e9"
Jul 17 14:28:15.257816 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:15.257779    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b" event={"ID":"356a5611-a29d-4a64-a840-e3ee09af90cf","Type":"ContainerStarted","Data":"0ee29e5a3117115af950fad9e7e09cf0945f1e47e1d13e6d89f9b1212e4e3a9d"}
Jul 17 14:28:15.259587 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:15.259565    2561 generic.go:356] "Generic (PLEG): container finished" podID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerID="ab5ff69fbcd5ebcdc655f85325e655e4a3e3cb7488b0534c2914a02cabba6bbb" exitCode=6
Jul 17 14:28:15.259692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:15.259649    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8" event={"ID":"37f6bc81-828b-4992-9273-2abd3c9ef683","Type":"ContainerDied","Data":"ab5ff69fbcd5ebcdc655f85325e655e4a3e3cb7488b0534c2914a02cabba6bbb"}
Jul 17 14:28:15.259737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:15.259708    2561 scope.go:117] "RemoveContainer" containerID="66e59f8db0aa8c454a2ca2bc04d0b8dc1900faa6dbfc3d6696ec16a6309da9c2"
Jul 17 14:28:15.259803 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:15.259788    2561 scope.go:117] "RemoveContainer" containerID="ab5ff69fbcd5ebcdc655f85325e655e4a3e3cb7488b0534c2914a02cabba6bbb"
Jul 17 14:28:15.260029 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:28:15.259984    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8_odh-ai-gateway-infra(37f6bc81-828b-4992-9273-2abd3c9ef683)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683"
Jul 17 14:28:15.279105 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:15.279062    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b" podStartSLOduration=17.279050361 podStartE2EDuration="17.279050361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:28:15.276994581 +0000 UTC m=+2580.983441414" watchObservedRunningTime="2026-07-17 14:28:15.279050361 +0000 UTC m=+2580.985497195"
Jul 17 14:28:16.269614 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:16.269586    2561 generic.go:356] "Generic (PLEG): container finished" podID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerID="0ee29e5a3117115af950fad9e7e09cf0945f1e47e1d13e6d89f9b1212e4e3a9d" exitCode=7
Jul 17 14:28:16.270034 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:16.269659    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b" event={"ID":"356a5611-a29d-4a64-a840-e3ee09af90cf","Type":"ContainerDied","Data":"0ee29e5a3117115af950fad9e7e09cf0945f1e47e1d13e6d89f9b1212e4e3a9d"}
Jul 17 14:28:16.270034 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:16.269715    2561 scope.go:117] "RemoveContainer" containerID="ceceb7d7bed8299178e4c2434ae5a2654501f26c40f284e3010333745bf8c8e9"
Jul 17 14:28:16.270034 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:16.269973    2561 scope.go:117] "RemoveContainer" containerID="0ee29e5a3117115af950fad9e7e09cf0945f1e47e1d13e6d89f9b1212e4e3a9d"
Jul 17 14:28:16.270238 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:28:16.270220    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b_odh-ai-gateway-infra(356a5611-a29d-4a64-a840-e3ee09af90cf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf"
Jul 17 14:28:16.271453 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:16.271436    2561 scope.go:117] "RemoveContainer" containerID="ab5ff69fbcd5ebcdc655f85325e655e4a3e3cb7488b0534c2914a02cabba6bbb"
Jul 17 14:28:16.271628 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:28:16.271611    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8_odh-ai-gateway-infra(37f6bc81-828b-4992-9273-2abd3c9ef683)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683"
Jul 17 14:28:16.289624 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:16.289603    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"]
Jul 17 14:28:17.423323 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:17.423298    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"
Jul 17 14:28:17.562624 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:17.562556    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/356a5611-a29d-4a64-a840-e3ee09af90cf-maas-api-service-ca\") pod \"356a5611-a29d-4a64-a840-e3ee09af90cf\" (UID: \"356a5611-a29d-4a64-a840-e3ee09af90cf\") "
Jul 17 14:28:17.562921 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:17.562897    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/356a5611-a29d-4a64-a840-e3ee09af90cf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "356a5611-a29d-4a64-a840-e3ee09af90cf" (UID: "356a5611-a29d-4a64-a840-e3ee09af90cf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:28:17.663414 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:17.663388    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/356a5611-a29d-4a64-a840-e3ee09af90cf-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:18.284981 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:18.284953    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"
Jul 17 14:28:18.285201 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:18.285022    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b" event={"ID":"356a5611-a29d-4a64-a840-e3ee09af90cf","Type":"ContainerDied","Data":"d58b3052baeccf7744051db58b3557a8ce324669933771bcc0c7de4a63dae4a0"}
Jul 17 14:28:18.285201 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:18.285084    2561 scope.go:117] "RemoveContainer" containerID="0ee29e5a3117115af950fad9e7e09cf0945f1e47e1d13e6d89f9b1212e4e3a9d"
Jul 17 14:28:18.324924 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:18.324898    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"]
Jul 17 14:28:18.336315 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:18.336288    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-jvz5b"]
Jul 17 14:28:18.773138 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:18.773111    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf" path="/var/lib/kubelet/pods/356a5611-a29d-4a64-a840-e3ee09af90cf/volumes"
Jul 17 14:28:18.907519 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:18.907485    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"]
Jul 17 14:28:18.907694 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:18.907660    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy" containerID="cri-o://2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b" gracePeriod=30
Jul 17 14:28:24.167318 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.167298    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:28:24.319567 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.319539    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmb5g\" (UniqueName: \"kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-kube-api-access-rmb5g\") pod \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") "
Jul 17 14:28:24.319743 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.319584    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-podinfo\") pod \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") "
Jul 17 14:28:24.319743 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.319618    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-data\") pod \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") "
Jul 17 14:28:24.319743 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.319646    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-socket\") pod \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") "
Jul 17 14:28:24.319743 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.319673    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-certs\") pod \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") "
Jul 17 14:28:24.319743 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.319710    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istiod-ca-cert\") pod \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") "
Jul 17 14:28:24.320049 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.319761    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-credential-socket\") pod \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") "
Jul 17 14:28:24.320049 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.319788    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-envoy\") pod \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") "
Jul 17 14:28:24.320049 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.319876    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-token\") pod \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\" (UID: \"ac86132f-6c04-4114-a3c6-f23ff4ea43ee\") "
Jul 17 14:28:24.320049 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.320001    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ac86132f-6c04-4114-a3c6-f23ff4ea43ee" (UID: "ac86132f-6c04-4114-a3c6-f23ff4ea43ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:28:24.320049 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.320043    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ac86132f-6c04-4114-a3c6-f23ff4ea43ee" (UID: "ac86132f-6c04-4114-a3c6-f23ff4ea43ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:28:24.320332 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.320143    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ac86132f-6c04-4114-a3c6-f23ff4ea43ee" (UID: "ac86132f-6c04-4114-a3c6-f23ff4ea43ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:28:24.320332 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.320244    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ac86132f-6c04-4114-a3c6-f23ff4ea43ee" (UID: "ac86132f-6c04-4114-a3c6-f23ff4ea43ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:28:24.320450 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.320428    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:24.320508 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.320457    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:24.320508 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.320474    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:24.320899 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.320865    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "ac86132f-6c04-4114-a3c6-f23ff4ea43ee" (UID: "ac86132f-6c04-4114-a3c6-f23ff4ea43ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:28:24.322278 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.321458    2561 generic.go:356] "Generic (PLEG): container finished" podID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerID="2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b" exitCode=0
Jul 17 14:28:24.322278 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.321603    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" event={"ID":"ac86132f-6c04-4114-a3c6-f23ff4ea43ee","Type":"ContainerDied","Data":"2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b"}
Jul 17 14:28:24.322278 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.321693    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv" event={"ID":"ac86132f-6c04-4114-a3c6-f23ff4ea43ee","Type":"ContainerDied","Data":"d2f1fc5e4734d205f5280223424387cd817ced6a8cdeed4f97be022c646780a5"}
Jul 17 14:28:24.322278 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.321726    2561 scope.go:117] "RemoveContainer" containerID="2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b"
Jul 17 14:28:24.322278 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.321904    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"
Jul 17 14:28:24.322596 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.322320    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-kube-api-access-rmb5g" (OuterVolumeSpecName: "kube-api-access-rmb5g") pod "ac86132f-6c04-4114-a3c6-f23ff4ea43ee" (UID: "ac86132f-6c04-4114-a3c6-f23ff4ea43ee"). InnerVolumeSpecName "kube-api-access-rmb5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:28:24.322596 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.322576    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ac86132f-6c04-4114-a3c6-f23ff4ea43ee" (UID: "ac86132f-6c04-4114-a3c6-f23ff4ea43ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:28:24.323952 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.323368    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "ac86132f-6c04-4114-a3c6-f23ff4ea43ee" (UID: "ac86132f-6c04-4114-a3c6-f23ff4ea43ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:28:24.323952 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.323481    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ac86132f-6c04-4114-a3c6-f23ff4ea43ee" (UID: "ac86132f-6c04-4114-a3c6-f23ff4ea43ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:28:24.344563 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.344541    2561 scope.go:117] "RemoveContainer" containerID="8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb"
Jul 17 14:28:24.353765 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.353740    2561 scope.go:117] "RemoveContainer" containerID="2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b"
Jul 17 14:28:24.354034 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:28:24.354011    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b\": container with ID starting with 2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b not found: ID does not exist" containerID="2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b"
Jul 17 14:28:24.354112 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.354042    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b"} err="failed to get container status \"2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b\": rpc error: code = NotFound desc = could not find container \"2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b\": container with ID starting with 2f23e4edf5646902c5e9da5ce8db142408b9756ea30180d3815c1bfc2b7d464b not found: ID does not exist"
Jul 17 14:28:24.354112 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.354069    2561 scope.go:117] "RemoveContainer" containerID="8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb"
Jul 17 14:28:24.354343 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:28:24.354326    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb\": container with ID starting with 8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb not found: ID does not exist" containerID="8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb"
Jul 17 14:28:24.354380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.354349    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb"} err="failed to get container status \"8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb\": rpc error: code = NotFound desc = could not find container \"8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb\": container with ID starting with 8b16f9bc0eac89969899188ae6f16e01a0610a3ae628a51b5eb763f61febc9cb not found: ID does not exist"
Jul 17 14:28:24.421251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.421186    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:24.421251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.421211    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmb5g\" (UniqueName: \"kubernetes.io/projected/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-kube-api-access-rmb5g\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:24.421251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.421221    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:24.421251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.421230    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:24.421251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.421238    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:24.421251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.421246    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac86132f-6c04-4114-a3c6-f23ff4ea43ee-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:24.662457 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.662424    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"]
Jul 17 14:28:24.669597 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.669571    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-73fe1bea-openshift-default-56cdb67459-2w4zv"]
Jul 17 14:28:24.772376 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:24.772302    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" path="/var/lib/kubelet/pods/ac86132f-6c04-4114-a3c6-f23ff4ea43ee/volumes"
Jul 17 14:28:29.765922 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:29.765891    2561 scope.go:117] "RemoveContainer" containerID="ab5ff69fbcd5ebcdc655f85325e655e4a3e3cb7488b0534c2914a02cabba6bbb"
Jul 17 14:28:30.364381 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:30.364351    2561 generic.go:356] "Generic (PLEG): container finished" podID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerID="a157b9cb387929189997233bd872bc114648f1881e1880c265a9ed86a2ac8f30" exitCode=6
Jul 17 14:28:30.364493 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:30.364424    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8" event={"ID":"37f6bc81-828b-4992-9273-2abd3c9ef683","Type":"ContainerDied","Data":"a157b9cb387929189997233bd872bc114648f1881e1880c265a9ed86a2ac8f30"}
Jul 17 14:28:30.364493 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:30.364483    2561 scope.go:117] "RemoveContainer" containerID="ab5ff69fbcd5ebcdc655f85325e655e4a3e3cb7488b0534c2914a02cabba6bbb"
Jul 17 14:28:30.364822 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:30.364796    2561 scope.go:117] "RemoveContainer" containerID="a157b9cb387929189997233bd872bc114648f1881e1880c265a9ed86a2ac8f30"
Jul 17 14:28:30.365076 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:28:30.365040    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8_odh-ai-gateway-infra(37f6bc81-828b-4992-9273-2abd3c9ef683)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683"
Jul 17 14:28:31.407188 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:31.407160    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"]
Jul 17 14:28:31.552672 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:31.552650    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"
Jul 17 14:28:31.688909 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:31.688876    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37f6bc81-828b-4992-9273-2abd3c9ef683-maas-api-service-ca\") pod \"37f6bc81-828b-4992-9273-2abd3c9ef683\" (UID: \"37f6bc81-828b-4992-9273-2abd3c9ef683\") "
Jul 17 14:28:31.689211 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:31.689189    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f6bc81-828b-4992-9273-2abd3c9ef683-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37f6bc81-828b-4992-9273-2abd3c9ef683" (UID: "37f6bc81-828b-4992-9273-2abd3c9ef683"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:28:31.689315 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:31.689302    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37f6bc81-828b-4992-9273-2abd3c9ef683-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:28:32.377121 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:32.377086    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8" event={"ID":"37f6bc81-828b-4992-9273-2abd3c9ef683","Type":"ContainerDied","Data":"68838ffc5ca6b0ab0d4e812c5e5a04b7dca71c37c19a250a461b82116ad0d128"}
Jul 17 14:28:32.377121 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:32.377119    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"
Jul 17 14:28:32.377354 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:32.377147    2561 scope.go:117] "RemoveContainer" containerID="a157b9cb387929189997233bd872bc114648f1881e1880c265a9ed86a2ac8f30"
Jul 17 14:28:32.403947 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:32.403921    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"]
Jul 17 14:28:32.419719 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:32.419698    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-398a9b19-cl6c8"]
Jul 17 14:28:32.778748 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:28:32.778713    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683" path="/var/lib/kubelet/pods/37f6bc81-828b-4992-9273-2abd3c9ef683/volumes"
Jul 17 14:30:12.574486 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.574400    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"]
Jul 17 14:30:12.575074 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575055    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerName="revoke-keys"
Jul 17 14:30:12.575162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575075    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerName="revoke-keys"
Jul 17 14:30:12.575162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575093    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy"
Jul 17 14:30:12.575162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575102    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy"
Jul 17 14:30:12.575162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575124    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerName="revoke-keys"
Jul 17 14:30:12.575162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575134    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerName="revoke-keys"
Jul 17 14:30:12.575162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575147    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerName="revoke-keys"
Jul 17 14:30:12.575162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575156    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerName="revoke-keys"
Jul 17 14:30:12.575507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575178    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerName="revoke-keys"
Jul 17 14:30:12.575507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575187    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerName="revoke-keys"
Jul 17 14:30:12.575507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575199    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerName="revoke-keys"
Jul 17 14:30:12.575507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575206    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerName="revoke-keys"
Jul 17 14:30:12.575507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575309    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerName="revoke-keys"
Jul 17 14:30:12.575507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575322    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerName="revoke-keys"
Jul 17 14:30:12.575507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575331    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy"
Jul 17 14:30:12.575507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575342    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy"
Jul 17 14:30:12.575507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575354    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerName="revoke-keys"
Jul 17 14:30:12.575507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.575363    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerName="revoke-keys"
Jul 17 14:30:12.578745 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.578724    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.581475 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.581453    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8ed37e19-openshift-default-dockercfg-dzr5j\""
Jul 17 14:30:12.600155 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.600126    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"]
Jul 17 14:30:12.632385 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.632353    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-credential-socket\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.632385 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.632389    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-istio-token\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.632630 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.632419    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-envoy\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.632630 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.632445    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/681e9e01-8939-49db-ad0e-e1be58f44977-istiod-ca-cert\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.632630 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.632531    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-data\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.632630 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.632615    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkm4m\" (UniqueName: \"kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-kube-api-access-pkm4m\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.632868 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.632644    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-certs\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.632868 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.632675    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-socket\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.632868 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.632701    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/681e9e01-8939-49db-ad0e-e1be58f44977-istio-podinfo\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.733721 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.733687    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkm4m\" (UniqueName: \"kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-kube-api-access-pkm4m\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.733891 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.733732    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-certs\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.733891 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.733855    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-socket\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.733975 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.733899    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/681e9e01-8939-49db-ad0e-e1be58f44977-istio-podinfo\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.734055 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.733985    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-credential-socket\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.734111 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.734063    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-istio-token\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.734157 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.734124    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-envoy\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.734214 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.734152    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-certs\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.734214 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.734157    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/681e9e01-8939-49db-ad0e-e1be58f44977-istiod-ca-cert\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.734314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.734217    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-data\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.734314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.734239    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-socket\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.734418 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.734367    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-credential-socket\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.734526 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.734508    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-data\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.734828 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.734802    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/681e9e01-8939-49db-ad0e-e1be58f44977-istiod-ca-cert\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.736671 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.736646    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-envoy\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.736826 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.736808    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/681e9e01-8939-49db-ad0e-e1be58f44977-istio-podinfo\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.755112 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.755087    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-istio-token\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.757310 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.757285    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkm4m\" (UniqueName: \"kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-kube-api-access-pkm4m\") pod \"e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") " pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:12.889664 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:12.889494    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:13.069053 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:13.068980    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"]
Jul 17 14:30:13.069601 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:30:13.069574    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod681e9e01_8939_49db_ad0e_e1be58f44977.slice/crio-2b1e2acf1708838e11523332683ec589baa89deafdce289a891b60b9949ff115 WatchSource:0}: Error finding container 2b1e2acf1708838e11523332683ec589baa89deafdce289a891b60b9949ff115: Status 404 returned error can't find the container with id 2b1e2acf1708838e11523332683ec589baa89deafdce289a891b60b9949ff115
Jul 17 14:30:13.071803 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:13.071769    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:30:13.071884 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:13.071849    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:30:13.071924 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:13.071891    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:30:13.929570 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:13.929537    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" event={"ID":"681e9e01-8939-49db-ad0e-e1be58f44977","Type":"ContainerStarted","Data":"62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590"}
Jul 17 14:30:13.929953 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:13.929599    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" event={"ID":"681e9e01-8939-49db-ad0e-e1be58f44977","Type":"ContainerStarted","Data":"2b1e2acf1708838e11523332683ec589baa89deafdce289a891b60b9949ff115"}
Jul 17 14:30:13.966838 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:13.966778    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" podStartSLOduration=1.966758339 podStartE2EDuration="1.966758339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:30:13.964525749 +0000 UTC m=+2699.670972599" watchObservedRunningTime="2026-07-17 14:30:13.966758339 +0000 UTC m=+2699.673205173"
Jul 17 14:30:14.890628 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:14.890592    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:14.896696 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:14.896672    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:14.936316 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:14.936292    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:14.937751 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:14.937728    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:28.006960 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.006921    2561 generic.go:356] "Generic (PLEG): container finished" podID="681e9e01-8939-49db-ad0e-e1be58f44977" containerID="62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590" exitCode=0
Jul 17 14:30:28.007402 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.006994    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" event={"ID":"681e9e01-8939-49db-ad0e-e1be58f44977","Type":"ContainerDied","Data":"62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590"}
Jul 17 14:30:28.007455 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.007426    2561 scope.go:117] "RemoveContainer" containerID="62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590"
Jul 17 14:30:28.008525 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.008499    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:30:28.008603 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.008567    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:30:28.008640 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.008599    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:30:28.683730 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.683699    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"]
Jul 17 14:30:28.684162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.684148    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerName="revoke-keys"
Jul 17 14:30:28.684162 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.684163    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerName="revoke-keys"
Jul 17 14:30:28.684266 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.684174    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy"
Jul 17 14:30:28.684266 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.684179    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac86132f-6c04-4114-a3c6-f23ff4ea43ee" containerName="istio-proxy"
Jul 17 14:30:28.684266 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.684240    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="356a5611-a29d-4a64-a840-e3ee09af90cf" containerName="revoke-keys"
Jul 17 14:30:28.684266 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.684252    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f6bc81-828b-4992-9273-2abd3c9ef683" containerName="revoke-keys"
Jul 17 14:30:28.686450 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.686429    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"
Jul 17 14:30:28.689221 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.689199    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:30:28.689319 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.689199    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:30:28.696614 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.696592    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"]
Jul 17 14:30:28.794074 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.794037    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/089e86bc-bb2e-4c53-8e17-6b86d716782f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8ed37e19-7t548\" (UID: \"089e86bc-bb2e-4c53-8e17-6b86d716782f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"
Jul 17 14:30:28.895113 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.895063    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/089e86bc-bb2e-4c53-8e17-6b86d716782f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8ed37e19-7t548\" (UID: \"089e86bc-bb2e-4c53-8e17-6b86d716782f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"
Jul 17 14:30:28.896163 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.896129    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/089e86bc-bb2e-4c53-8e17-6b86d716782f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8ed37e19-7t548\" (UID: \"089e86bc-bb2e-4c53-8e17-6b86d716782f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"
Jul 17 14:30:28.998333 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:28.998248    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"
Jul 17 14:30:29.014999 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:29.014968    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" event={"ID":"681e9e01-8939-49db-ad0e-e1be58f44977","Type":"ContainerStarted","Data":"ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e"}
Jul 17 14:30:29.210505 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:29.210479    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"]
Jul 17 14:30:29.212535 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:30:29.212503    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod089e86bc_bb2e_4c53_8e17_6b86d716782f.slice/crio-083cf0d7822f95b7011651544c874dd9bc4ab3cb9a46a4a043011740cb238b7e WatchSource:0}: Error finding container 083cf0d7822f95b7011651544c874dd9bc4ab3cb9a46a4a043011740cb238b7e: Status 404 returned error can't find the container with id 083cf0d7822f95b7011651544c874dd9bc4ab3cb9a46a4a043011740cb238b7e
Jul 17 14:30:29.890086 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:29.889976    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:29.890965 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:29.890944    2561 patch_prober.go:28] interesting pod/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Jul 17 14:30:29.891036 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:29.890993    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Jul 17 14:30:29.936805 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:29.936773    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:30.023522 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:30.023466    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548" event={"ID":"089e86bc-bb2e-4c53-8e17-6b86d716782f","Type":"ContainerStarted","Data":"83068dfcf5875c49180391cd0f8c3b8600ebdecc70b540b22ba42181a13cb027"}
Jul 17 14:30:30.023998 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:30.023547    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548" event={"ID":"089e86bc-bb2e-4c53-8e17-6b86d716782f","Type":"ContainerStarted","Data":"083cf0d7822f95b7011651544c874dd9bc4ab3cb9a46a4a043011740cb238b7e"}
Jul 17 14:30:30.051163 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:30.051086    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548" podStartSLOduration=2.051065865 podStartE2EDuration="2.051065865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:30:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:30:30.046714758 +0000 UTC m=+2715.753161592" watchObservedRunningTime="2026-07-17 14:30:30.051065865 +0000 UTC m=+2715.757512701"
Jul 17 14:30:30.890373 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:30.890339    2561 patch_prober.go:28] interesting pod/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Jul 17 14:30:30.890550 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:30.890406    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Jul 17 14:30:31.049648 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:31.048714    2561 generic.go:356] "Generic (PLEG): container finished" podID="089e86bc-bb2e-4c53-8e17-6b86d716782f" containerID="83068dfcf5875c49180391cd0f8c3b8600ebdecc70b540b22ba42181a13cb027" exitCode=0
Jul 17 14:30:31.049648 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:31.048884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548" event={"ID":"089e86bc-bb2e-4c53-8e17-6b86d716782f","Type":"ContainerDied","Data":"83068dfcf5875c49180391cd0f8c3b8600ebdecc70b540b22ba42181a13cb027"}
Jul 17 14:30:31.890782 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:31.890748    2561 patch_prober.go:28] interesting pod/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Jul 17 14:30:31.890954 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:31.890803    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Jul 17 14:30:32.248843 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:32.248806    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"
Jul 17 14:30:32.347506 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:32.347473    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/089e86bc-bb2e-4c53-8e17-6b86d716782f-maas-api-service-ca\") pod \"089e86bc-bb2e-4c53-8e17-6b86d716782f\" (UID: \"089e86bc-bb2e-4c53-8e17-6b86d716782f\") "
Jul 17 14:30:32.347844 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:32.347811    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/089e86bc-bb2e-4c53-8e17-6b86d716782f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "089e86bc-bb2e-4c53-8e17-6b86d716782f" (UID: "089e86bc-bb2e-4c53-8e17-6b86d716782f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:30:32.448736 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:32.448706    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/089e86bc-bb2e-4c53-8e17-6b86d716782f-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:30:32.890293 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:32.890255    2561 patch_prober.go:28] interesting pod/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Jul 17 14:30:32.890469 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:32.890313    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Jul 17 14:30:33.061257 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:33.061178    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"
Jul 17 14:30:33.061257 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:33.061218    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548" event={"ID":"089e86bc-bb2e-4c53-8e17-6b86d716782f","Type":"ContainerDied","Data":"083cf0d7822f95b7011651544c874dd9bc4ab3cb9a46a4a043011740cb238b7e"}
Jul 17 14:30:33.061443 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:33.061263    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="083cf0d7822f95b7011651544c874dd9bc4ab3cb9a46a4a043011740cb238b7e"
Jul 17 14:30:33.894635 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:33.894608    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:33.895596 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:33.895578    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:39.086529 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.086497    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"]
Jul 17 14:30:39.086960 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.086946    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="089e86bc-bb2e-4c53-8e17-6b86d716782f" containerName="revoke-keys"
Jul 17 14:30:39.087036 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.086962    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="089e86bc-bb2e-4c53-8e17-6b86d716782f" containerName="revoke-keys"
Jul 17 14:30:39.087083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.087044    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="089e86bc-bb2e-4c53-8e17-6b86d716782f" containerName="revoke-keys"
Jul 17 14:30:39.089227 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.089203    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"
Jul 17 14:30:39.091903 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.091883    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:30:39.092067 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.092043    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:30:39.097758 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.097737    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"]
Jul 17 14:30:39.115364 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.115342    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-pv869\" (UID: \"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"
Jul 17 14:30:39.216865 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.216823    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-pv869\" (UID: \"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"
Jul 17 14:30:39.217685 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.217657    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-73fe1bea-pv869\" (UID: \"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"
Jul 17 14:30:39.400905 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.400819    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"
Jul 17 14:30:39.537703 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:39.537674    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"]
Jul 17 14:30:39.540055 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:30:39.539992    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b1b1e3_5ca3_4b6f_80bb_e7ac105debe7.slice/crio-1570e268aa8681cfc0566fef6d2d26a0d7c0d7d83f5a7907c72c37b7b4448ef0 WatchSource:0}: Error finding container 1570e268aa8681cfc0566fef6d2d26a0d7c0d7d83f5a7907c72c37b7b4448ef0: Status 404 returned error can't find the container with id 1570e268aa8681cfc0566fef6d2d26a0d7c0d7d83f5a7907c72c37b7b4448ef0
Jul 17 14:30:40.101586 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:40.101489    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" event={"ID":"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7","Type":"ContainerStarted","Data":"d1ca7a8f13d3d7c4a731581ab86c00b48ce4f78642b64871375b1d9105c61006"}
Jul 17 14:30:40.101586 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:40.101574    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" event={"ID":"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7","Type":"ContainerStarted","Data":"1570e268aa8681cfc0566fef6d2d26a0d7c0d7d83f5a7907c72c37b7b4448ef0"}
Jul 17 14:30:40.120016 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:40.119956    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" podStartSLOduration=1.119938192 podStartE2EDuration="1.119938192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:30:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:30:40.117953473 +0000 UTC m=+2725.824400309" watchObservedRunningTime="2026-07-17 14:30:40.119938192 +0000 UTC m=+2725.826385026"
Jul 17 14:30:41.108954 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:41.108923    2561 generic.go:356] "Generic (PLEG): container finished" podID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerID="d1ca7a8f13d3d7c4a731581ab86c00b48ce4f78642b64871375b1d9105c61006" exitCode=6
Jul 17 14:30:41.109375 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:41.108972    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" event={"ID":"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7","Type":"ContainerDied","Data":"d1ca7a8f13d3d7c4a731581ab86c00b48ce4f78642b64871375b1d9105c61006"}
Jul 17 14:30:41.109375 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:41.109198    2561 scope.go:117] "RemoveContainer" containerID="d1ca7a8f13d3d7c4a731581ab86c00b48ce4f78642b64871375b1d9105c61006"
Jul 17 14:30:42.116412 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:42.116321    2561 generic.go:356] "Generic (PLEG): container finished" podID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerID="963825495c404e57b1aed7e7d5037b8def050243df0ee5f2c48badfba3f47155" exitCode=6
Jul 17 14:30:42.116412 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:42.116400    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" event={"ID":"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7","Type":"ContainerDied","Data":"963825495c404e57b1aed7e7d5037b8def050243df0ee5f2c48badfba3f47155"}
Jul 17 14:30:42.116885 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:42.116456    2561 scope.go:117] "RemoveContainer" containerID="d1ca7a8f13d3d7c4a731581ab86c00b48ce4f78642b64871375b1d9105c61006"
Jul 17 14:30:42.116885 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:42.116623    2561 scope.go:117] "RemoveContainer" containerID="963825495c404e57b1aed7e7d5037b8def050243df0ee5f2c48badfba3f47155"
Jul 17 14:30:42.116972 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:30:42.116908    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-pv869_odh-ai-gateway-infra(d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7"
Jul 17 14:30:43.124388 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:43.124357    2561 scope.go:117] "RemoveContainer" containerID="963825495c404e57b1aed7e7d5037b8def050243df0ee5f2c48badfba3f47155"
Jul 17 14:30:43.124970 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:30:43.124637    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-pv869_odh-ai-gateway-infra(d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7"
Jul 17 14:30:43.776795 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:43.776754    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"]
Jul 17 14:30:43.779919 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:43.779892    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8ed37e19-7t548"]
Jul 17 14:30:44.779714 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:44.779673    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="089e86bc-bb2e-4c53-8e17-6b86d716782f" path="/var/lib/kubelet/pods/089e86bc-bb2e-4c53-8e17-6b86d716782f/volumes"
Jul 17 14:30:50.846203 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:50.846161    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"]
Jul 17 14:30:50.846704 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:50.846447    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy" containerID="cri-o://ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e" gracePeriod=30
Jul 17 14:30:56.119485 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.119421    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:56.194304 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194276    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-istio-token\") pod \"681e9e01-8939-49db-ad0e-e1be58f44977\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") "
Jul 17 14:30:56.194304 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194309    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkm4m\" (UniqueName: \"kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-kube-api-access-pkm4m\") pod \"681e9e01-8939-49db-ad0e-e1be58f44977\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") "
Jul 17 14:30:56.194560 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194361    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-certs\") pod \"681e9e01-8939-49db-ad0e-e1be58f44977\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") "
Jul 17 14:30:56.194560 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194423    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-socket\") pod \"681e9e01-8939-49db-ad0e-e1be58f44977\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") "
Jul 17 14:30:56.194560 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194478    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-data\") pod \"681e9e01-8939-49db-ad0e-e1be58f44977\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") "
Jul 17 14:30:56.194560 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194529    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/681e9e01-8939-49db-ad0e-e1be58f44977-istio-podinfo\") pod \"681e9e01-8939-49db-ad0e-e1be58f44977\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") "
Jul 17 14:30:56.194776 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194583    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-envoy\") pod \"681e9e01-8939-49db-ad0e-e1be58f44977\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") "
Jul 17 14:30:56.194776 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194622    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/681e9e01-8939-49db-ad0e-e1be58f44977-istiod-ca-cert\") pod \"681e9e01-8939-49db-ad0e-e1be58f44977\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") "
Jul 17 14:30:56.194776 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194648    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-credential-socket\") pod \"681e9e01-8939-49db-ad0e-e1be58f44977\" (UID: \"681e9e01-8939-49db-ad0e-e1be58f44977\") "
Jul 17 14:30:56.194776 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194699    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "681e9e01-8939-49db-ad0e-e1be58f44977" (UID: "681e9e01-8939-49db-ad0e-e1be58f44977"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:30:56.194776 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194725    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "681e9e01-8939-49db-ad0e-e1be58f44977" (UID: "681e9e01-8939-49db-ad0e-e1be58f44977"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:30:56.195125 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.194996    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:30:56.195125 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.195044    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:30:56.195125 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.195069    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/681e9e01-8939-49db-ad0e-e1be58f44977-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "681e9e01-8939-49db-ad0e-e1be58f44977" (UID: "681e9e01-8939-49db-ad0e-e1be58f44977"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:30:56.195373 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.195260    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "681e9e01-8939-49db-ad0e-e1be58f44977" (UID: "681e9e01-8939-49db-ad0e-e1be58f44977"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:30:56.195953 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.195919    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-data" (OuterVolumeSpecName: "istio-data") pod "681e9e01-8939-49db-ad0e-e1be58f44977" (UID: "681e9e01-8939-49db-ad0e-e1be58f44977"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:30:56.197112 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.197081    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/681e9e01-8939-49db-ad0e-e1be58f44977-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "681e9e01-8939-49db-ad0e-e1be58f44977" (UID: "681e9e01-8939-49db-ad0e-e1be58f44977"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:30:56.197267 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.197246    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-kube-api-access-pkm4m" (OuterVolumeSpecName: "kube-api-access-pkm4m") pod "681e9e01-8939-49db-ad0e-e1be58f44977" (UID: "681e9e01-8939-49db-ad0e-e1be58f44977"). InnerVolumeSpecName "kube-api-access-pkm4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:30:56.197343 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.197276    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-istio-token" (OuterVolumeSpecName: "istio-token") pod "681e9e01-8939-49db-ad0e-e1be58f44977" (UID: "681e9e01-8939-49db-ad0e-e1be58f44977"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:30:56.197544 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.197506    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "681e9e01-8939-49db-ad0e-e1be58f44977" (UID: "681e9e01-8939-49db-ad0e-e1be58f44977"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:30:56.197710 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.197690    2561 generic.go:356] "Generic (PLEG): container finished" podID="681e9e01-8939-49db-ad0e-e1be58f44977" containerID="ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e" exitCode=0
Jul 17 14:30:56.197770 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.197748    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"
Jul 17 14:30:56.197770 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.197756    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" event={"ID":"681e9e01-8939-49db-ad0e-e1be58f44977","Type":"ContainerDied","Data":"ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e"}
Jul 17 14:30:56.197846 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.197799    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w" event={"ID":"681e9e01-8939-49db-ad0e-e1be58f44977","Type":"ContainerDied","Data":"2b1e2acf1708838e11523332683ec589baa89deafdce289a891b60b9949ff115"}
Jul 17 14:30:56.197846 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.197819    2561 scope.go:117] "RemoveContainer" containerID="ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e"
Jul 17 14:30:56.218652 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.218628    2561 scope.go:117] "RemoveContainer" containerID="62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590"
Jul 17 14:30:56.227157 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.227138    2561 scope.go:117] "RemoveContainer" containerID="ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e"
Jul 17 14:30:56.227457 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:30:56.227435    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e\": container with ID starting with ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e not found: ID does not exist" containerID="ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e"
Jul 17 14:30:56.227534 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.227470    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e"} err="failed to get container status \"ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e\": rpc error: code = NotFound desc = could not find container \"ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e\": container with ID starting with ea50eb2a714a8907c5825c0bfce689f55c63e9c9e080d4a3f61f1e21d844bc1e not found: ID does not exist"
Jul 17 14:30:56.227534 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.227502    2561 scope.go:117] "RemoveContainer" containerID="62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590"
Jul 17 14:30:56.227762 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:30:56.227745    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590\": container with ID starting with 62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590 not found: ID does not exist" containerID="62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590"
Jul 17 14:30:56.227802 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.227785    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590"} err="failed to get container status \"62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590\": rpc error: code = NotFound desc = could not find container \"62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590\": container with ID starting with 62cf7ec9a2d9acb83d8db47e000eeb534b05f401bb70836c6f4946f4b9ac4590 not found: ID does not exist"
Jul 17 14:30:56.236622 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.236520    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"]
Jul 17 14:30:56.238741 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.238718    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8ed37e19-openshift-default-74bbb849f-nm78w"]
Jul 17 14:30:56.296355 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.296320    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:30:56.296355 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.296348    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/681e9e01-8939-49db-ad0e-e1be58f44977-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:30:56.296355 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.296360    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:30:56.296603 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.296374    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/681e9e01-8939-49db-ad0e-e1be58f44977-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:30:56.296603 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.296385    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/681e9e01-8939-49db-ad0e-e1be58f44977-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:30:56.296603 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.296397    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:30:56.296603 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.296408    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkm4m\" (UniqueName: \"kubernetes.io/projected/681e9e01-8939-49db-ad0e-e1be58f44977-kube-api-access-pkm4m\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:30:56.779530 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:56.779489    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" path="/var/lib/kubelet/pods/681e9e01-8939-49db-ad0e-e1be58f44977/volumes"
Jul 17 14:30:57.765788 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:57.765753    2561 scope.go:117] "RemoveContainer" containerID="963825495c404e57b1aed7e7d5037b8def050243df0ee5f2c48badfba3f47155"
Jul 17 14:30:58.213770 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:58.213737    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" event={"ID":"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7","Type":"ContainerStarted","Data":"8d9eb14d204d7b8ca92b9d6cfd0e927a1bbfe9369b4203156a2724aabe384485"}
Jul 17 14:30:59.221512 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:59.221477    2561 generic.go:356] "Generic (PLEG): container finished" podID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerID="8d9eb14d204d7b8ca92b9d6cfd0e927a1bbfe9369b4203156a2724aabe384485" exitCode=6
Jul 17 14:30:59.221946 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:59.221531    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" event={"ID":"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7","Type":"ContainerDied","Data":"8d9eb14d204d7b8ca92b9d6cfd0e927a1bbfe9369b4203156a2724aabe384485"}
Jul 17 14:30:59.221946 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:59.221594    2561 scope.go:117] "RemoveContainer" containerID="963825495c404e57b1aed7e7d5037b8def050243df0ee5f2c48badfba3f47155"
Jul 17 14:30:59.221946 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:59.221899    2561 scope.go:117] "RemoveContainer" containerID="8d9eb14d204d7b8ca92b9d6cfd0e927a1bbfe9369b4203156a2724aabe384485"
Jul 17 14:30:59.222173 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:30:59.222152    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-73fe1bea-pv869_odh-ai-gateway-infra(d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7"
Jul 17 14:30:59.274555 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:30:59.274520    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"]
Jul 17 14:31:00.373719 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:00.373694    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"
Jul 17 14:31:00.536170 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:00.536078    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7-maas-api-service-ca\") pod \"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7\" (UID: \"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7\") "
Jul 17 14:31:00.536505 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:00.536477    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" (UID: "d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:31:00.637146 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:00.637109    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:31:01.235710 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:01.235675    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869" event={"ID":"d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7","Type":"ContainerDied","Data":"1570e268aa8681cfc0566fef6d2d26a0d7c0d7d83f5a7907c72c37b7b4448ef0"}
Jul 17 14:31:01.235907 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:01.235740    2561 scope.go:117] "RemoveContainer" containerID="8d9eb14d204d7b8ca92b9d6cfd0e927a1bbfe9369b4203156a2724aabe384485"
Jul 17 14:31:01.235907 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:01.235692    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"
Jul 17 14:31:01.255163 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:01.255137    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"]
Jul 17 14:31:01.260477 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:01.260455    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-73fe1bea-pv869"]
Jul 17 14:31:02.772934 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:02.772901    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" path="/var/lib/kubelet/pods/d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7/volumes"
Jul 17 14:31:11.671703 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.671658    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"]
Jul 17 14:31:11.672303 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.672283    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerName="revoke-keys"
Jul 17 14:31:11.672396 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.672306    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerName="revoke-keys"
Jul 17 14:31:11.672396 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.672332    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy"
Jul 17 14:31:11.672396 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.672340    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy"
Jul 17 14:31:11.672396 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.672382    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerName="revoke-keys"
Jul 17 14:31:11.672396 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.672391    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerName="revoke-keys"
Jul 17 14:31:11.672654 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.672479    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerName="revoke-keys"
Jul 17 14:31:11.672654 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.672491    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy"
Jul 17 14:31:11.672654 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.672505    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerName="revoke-keys"
Jul 17 14:31:11.677308 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.677285    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.681602 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.681577    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f387e370-openshift-default-dockercfg-8fth9\""
Jul 17 14:31:11.689611 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.689583    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"]
Jul 17 14:31:11.841067 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.841024    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-podinfo\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.841267 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.841094    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-certs\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.841267 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.841116    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-data\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.841267 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.841132    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-credential-socket\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.841267 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.841149    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istiod-ca-cert\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.841267 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.841170    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lghjq\" (UniqueName: \"kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-kube-api-access-lghjq\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.841267 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.841216    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-envoy\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.841590 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.841279    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-token\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.841590 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.841298    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-socket\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.942195 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.942096    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-podinfo\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.942372 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.942207    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-certs\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.942372 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.942240    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-data\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.942372 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.942270    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-credential-socket\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.942372 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.942297    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istiod-ca-cert\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.942372 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.942320    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lghjq\" (UniqueName: \"kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-kube-api-access-lghjq\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.942372 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.942342    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-envoy\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.942692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.942623    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-certs\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.943191 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.943161    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istiod-ca-cert\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.943397 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.943349    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-data\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.943397 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.942408    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-token\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.943577 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.943417    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-socket\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.944310 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.944287    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-socket\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.944688 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.944661    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-credential-socket\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.945380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.945360    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-podinfo\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.945573 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.945557    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-envoy\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.951986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.951963    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-token\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.953335 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.953311    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lghjq\" (UniqueName: \"kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-kube-api-access-lghjq\") pod \"e2e-mt-f387e370-openshift-default-947948c7f-dswmw\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") " pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:11.991117 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:11.991087    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:12.174032 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:12.172950    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"]
Jul 17 14:31:12.206789 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:31:12.202698    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94c7e93_075a_437e_a7f9_bbcf0d220feb.slice/crio-90fb9892eb956c51ec74f359e0f83e9dcf29b91894f05e6e7a1e2290a614195f WatchSource:0}: Error finding container 90fb9892eb956c51ec74f359e0f83e9dcf29b91894f05e6e7a1e2290a614195f: Status 404 returned error can't find the container with id 90fb9892eb956c51ec74f359e0f83e9dcf29b91894f05e6e7a1e2290a614195f
Jul 17 14:31:12.206789 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:12.205645    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:31:12.206789 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:12.205729    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:31:12.206789 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:12.205773    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:31:12.298647 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:12.298617    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" event={"ID":"a94c7e93-075a-437e-a7f9-bbcf0d220feb","Type":"ContainerStarted","Data":"03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d"}
Jul 17 14:31:12.298791 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:12.298677    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" event={"ID":"a94c7e93-075a-437e-a7f9-bbcf0d220feb","Type":"ContainerStarted","Data":"90fb9892eb956c51ec74f359e0f83e9dcf29b91894f05e6e7a1e2290a614195f"}
Jul 17 14:31:12.324657 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:12.324572    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" podStartSLOduration=1.324550508 podStartE2EDuration="1.324550508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:31:12.321260205 +0000 UTC m=+2758.027707040" watchObservedRunningTime="2026-07-17 14:31:12.324550508 +0000 UTC m=+2758.030997343"
Jul 17 14:31:12.991698 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:12.991664    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:13.996715 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:13.996686    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:14.312001 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:14.311916    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:14.312931 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:14.312912    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:25.385464 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:25.385373    2561 generic.go:356] "Generic (PLEG): container finished" podID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerID="03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d" exitCode=0
Jul 17 14:31:25.385464 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:25.385442    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" event={"ID":"a94c7e93-075a-437e-a7f9-bbcf0d220feb","Type":"ContainerDied","Data":"03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d"}
Jul 17 14:31:25.385891 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:25.385834    2561 scope.go:117] "RemoveContainer" containerID="03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d"
Jul 17 14:31:25.386780 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:25.386757    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:31:25.386834 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:25.386823    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:31:25.386869 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:25.386855    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:31:26.393205 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:26.393174    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" event={"ID":"a94c7e93-075a-437e-a7f9-bbcf0d220feb","Type":"ContainerStarted","Data":"98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53"}
Jul 17 14:31:26.991677 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:26.991643    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:26.992741 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:26.992706    2561 patch_prober.go:28] interesting pod/e2e-mt-f387e370-openshift-default-947948c7f-dswmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Jul 17 14:31:26.992876 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:26.992761    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Jul 17 14:31:27.992400 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:27.992360    2561 patch_prober.go:28] interesting pod/e2e-mt-f387e370-openshift-default-947948c7f-dswmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Jul 17 14:31:27.992798 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:27.992427    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Jul 17 14:31:28.716249 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.716221    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"]
Jul 17 14:31:28.716690 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.716676    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerName="revoke-keys"
Jul 17 14:31:28.716745 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.716692    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerName="revoke-keys"
Jul 17 14:31:28.716745 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.716708    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy"
Jul 17 14:31:28.716745 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.716714    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy"
Jul 17 14:31:28.716845 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.716778    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4b1b1e3-5ca3-4b6f-80bb-e7ac105debe7" containerName="revoke-keys"
Jul 17 14:31:28.716845 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.716787    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="681e9e01-8939-49db-ad0e-e1be58f44977" containerName="istio-proxy"
Jul 17 14:31:28.718984 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.718969    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.729826 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.729351    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f387e370\"/\"default-dockercfg-77b8t\""
Jul 17 14:31:28.729826 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.729613    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f387e370\"/\"kube-root-ca.crt\""
Jul 17 14:31:28.734570 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.734545    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"]
Jul 17 14:31:28.734869 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.734850    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f387e370\"/\"openshift-service-ca.crt\""
Jul 17 14:31:28.735259 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.735240    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f387e370\"/\"e2e-lifecycle-model-f387e370-kserve-self-signed-certs\""
Jul 17 14:31:28.806112 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.806082    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-dshm\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.806286 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.806119    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-home\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.806286 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.806143    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tmp-dir\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.806286 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.806231    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tls-certs\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.806286 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.806265    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-model-cache\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.806473 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.806292    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88snh\" (UniqueName: \"kubernetes.io/projected/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-kube-api-access-88snh\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.907193 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.907164    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tls-certs\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.907376 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.907211    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-model-cache\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.907376 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.907240    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88snh\" (UniqueName: \"kubernetes.io/projected/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-kube-api-access-88snh\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.907376 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.907269    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-dshm\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.907376 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.907288    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-home\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.907589 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.907547    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tmp-dir\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.907746 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.907715    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-model-cache\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.907846 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.907795    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-home\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.907983 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.907963    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tmp-dir\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.909648 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.909626    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-dshm\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.909791 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.909775    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tls-certs\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.916369 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.916346    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88snh\" (UniqueName: \"kubernetes.io/projected/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-kube-api-access-88snh\") pod \"e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") " pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:28.991526 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.991431    2561 patch_prober.go:28] interesting pod/e2e-mt-f387e370-openshift-default-947948c7f-dswmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Jul 17 14:31:28.991526 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:28.991489    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Jul 17 14:31:29.050290 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:29.050252    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:29.245086 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:29.241129    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"]
Jul 17 14:31:29.247139 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:31:29.245619    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76ed735f_9820_48aa_ac4d_4819d2e1b8fb.slice/crio-be8782b2eb75eac25ad49e7e2a2c859fec57d07bbd60ad922e39b5b3b6e41860 WatchSource:0}: Error finding container be8782b2eb75eac25ad49e7e2a2c859fec57d07bbd60ad922e39b5b3b6e41860: Status 404 returned error can't find the container with id be8782b2eb75eac25ad49e7e2a2c859fec57d07bbd60ad922e39b5b3b6e41860
Jul 17 14:31:29.312066 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:29.312040    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:29.411108 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:29.411071    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8" event={"ID":"76ed735f-9820-48aa-ac4d-4819d2e1b8fb","Type":"ContainerStarted","Data":"15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309"}
Jul 17 14:31:29.411287 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:29.411114    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8" event={"ID":"76ed735f-9820-48aa-ac4d-4819d2e1b8fb","Type":"ContainerStarted","Data":"be8782b2eb75eac25ad49e7e2a2c859fec57d07bbd60ad922e39b5b3b6e41860"}
Jul 17 14:31:29.432062 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:29.431960    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8" podStartSLOduration=1.4319084690000001 podStartE2EDuration="1.431908469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:31:29.430306081 +0000 UTC m=+2775.136752914" watchObservedRunningTime="2026-07-17 14:31:29.431908469 +0000 UTC m=+2775.138355306"
Jul 17 14:31:29.992275 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:29.992235    2561 patch_prober.go:28] interesting pod/e2e-mt-f387e370-openshift-default-947948c7f-dswmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Jul 17 14:31:29.992472 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:29.992309    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Jul 17 14:31:30.995429 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:30.995397    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:30.996654 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:30.996628    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:31:39.050589 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:39.050504    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:39.050589 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:39.050574    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:39.063096 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:39.063069    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:31:39.478607 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:31:39.478578    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:32:14.371326 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:14.371289    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"]
Jul 17 14:32:29.392244 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.392198    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8" podUID="76ed735f-9820-48aa-ac4d-4819d2e1b8fb" containerName="main" containerID="cri-o://15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309" gracePeriod=45
Jul 17 14:32:29.468398 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.468361    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8" podUID="76ed735f-9820-48aa-ac4d-4819d2e1b8fb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.94:8000/ready\": dial tcp 10.133.0.94:8000: connect: connection refused"
Jul 17 14:32:29.659465 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.659441    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:32:29.734311 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.734272    2561 generic.go:356] "Generic (PLEG): container finished" podID="76ed735f-9820-48aa-ac4d-4819d2e1b8fb" containerID="15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309" exitCode=0
Jul 17 14:32:29.734311 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.734303    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8" event={"ID":"76ed735f-9820-48aa-ac4d-4819d2e1b8fb","Type":"ContainerDied","Data":"15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309"}
Jul 17 14:32:29.734504 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.734320    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"
Jul 17 14:32:29.734504 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.734344    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8" event={"ID":"76ed735f-9820-48aa-ac4d-4819d2e1b8fb","Type":"ContainerDied","Data":"be8782b2eb75eac25ad49e7e2a2c859fec57d07bbd60ad922e39b5b3b6e41860"}
Jul 17 14:32:29.734504 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.734364    2561 scope.go:117] "RemoveContainer" containerID="15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309"
Jul 17 14:32:29.743116 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.743088    2561 scope.go:117] "RemoveContainer" containerID="15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309"
Jul 17 14:32:29.743370 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:32:29.743353    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309\": container with ID starting with 15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309 not found: ID does not exist" containerID="15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309"
Jul 17 14:32:29.743425 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.743381    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309"} err="failed to get container status \"15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309\": rpc error: code = NotFound desc = could not find container \"15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309\": container with ID starting with 15d96eb0337519321f83e26513db55ed0025c18883288c79dda7df03c5e67309 not found: ID does not exist"
Jul 17 14:32:29.771899 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.771873    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-home\") pod \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") "
Jul 17 14:32:29.772015 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.771903    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tmp-dir\") pod \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") "
Jul 17 14:32:29.772015 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.771921    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tls-certs\") pod \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") "
Jul 17 14:32:29.772015 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.771946    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88snh\" (UniqueName: \"kubernetes.io/projected/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-kube-api-access-88snh\") pod \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") "
Jul 17 14:32:29.772015 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.771972    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-model-cache\") pod \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") "
Jul 17 14:32:29.772238 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.772141    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-dshm\") pod \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\" (UID: \"76ed735f-9820-48aa-ac4d-4819d2e1b8fb\") "
Jul 17 14:32:29.772238 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.772141    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "76ed735f-9820-48aa-ac4d-4819d2e1b8fb" (UID: "76ed735f-9820-48aa-ac4d-4819d2e1b8fb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:32:29.772238 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.772163    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-home" (OuterVolumeSpecName: "home") pod "76ed735f-9820-48aa-ac4d-4819d2e1b8fb" (UID: "76ed735f-9820-48aa-ac4d-4819d2e1b8fb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:32:29.772414 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.772390    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-model-cache" (OuterVolumeSpecName: "model-cache") pod "76ed735f-9820-48aa-ac4d-4819d2e1b8fb" (UID: "76ed735f-9820-48aa-ac4d-4819d2e1b8fb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:32:29.772506 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.772490    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-home\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:32:29.772565 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.772512    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tmp-dir\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:32:29.772565 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.772527    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-model-cache\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:32:29.774292 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.774269    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "76ed735f-9820-48aa-ac4d-4819d2e1b8fb" (UID: "76ed735f-9820-48aa-ac4d-4819d2e1b8fb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 14:32:29.774385 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.774312    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-dshm" (OuterVolumeSpecName: "dshm") pod "76ed735f-9820-48aa-ac4d-4819d2e1b8fb" (UID: "76ed735f-9820-48aa-ac4d-4819d2e1b8fb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:32:29.774433 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.774381    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-kube-api-access-88snh" (OuterVolumeSpecName: "kube-api-access-88snh") pod "76ed735f-9820-48aa-ac4d-4819d2e1b8fb" (UID: "76ed735f-9820-48aa-ac4d-4819d2e1b8fb"). InnerVolumeSpecName "kube-api-access-88snh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:32:29.873893 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.873866    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-tls-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:32:29.873893 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.873891    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88snh\" (UniqueName: \"kubernetes.io/projected/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-kube-api-access-88snh\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:32:29.874097 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:29.873899    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76ed735f-9820-48aa-ac4d-4819d2e1b8fb-dshm\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:32:30.065908 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:30.065874    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"]
Jul 17 14:32:30.073931 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:30.073908    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f387e370/e2e-lifecycle-model-f387e370-kserve-7b54d98d5f-lp4n8"]
Jul 17 14:32:30.771646 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:32:30.771613    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76ed735f-9820-48aa-ac4d-4819d2e1b8fb" path="/var/lib/kubelet/pods/76ed735f-9820-48aa-ac4d-4819d2e1b8fb/volumes"
Jul 17 14:33:08.255394 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:08.255305    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"]
Jul 17 14:33:08.255977 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:08.255549    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy" containerID="cri-o://98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53" gracePeriod=30
Jul 17 14:33:13.542404 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.542372    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:33:13.697987 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.697948    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-envoy\") pod \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") "
Jul 17 14:33:13.698213 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698044    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lghjq\" (UniqueName: \"kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-kube-api-access-lghjq\") pod \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") "
Jul 17 14:33:13.698213 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698104    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-certs\") pod \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") "
Jul 17 14:33:13.698213 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698130    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-token\") pod \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") "
Jul 17 14:33:13.698213 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698156    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-credential-socket\") pod \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") "
Jul 17 14:33:13.698213 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698191    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-data\") pod \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") "
Jul 17 14:33:13.698481 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698255    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-socket\") pod \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") "
Jul 17 14:33:13.698481 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698300    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-podinfo\") pod \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") "
Jul 17 14:33:13.698481 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698337    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istiod-ca-cert\") pod \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\" (UID: \"a94c7e93-075a-437e-a7f9-bbcf0d220feb\") "
Jul 17 14:33:13.698481 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698365    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a94c7e93-075a-437e-a7f9-bbcf0d220feb" (UID: "a94c7e93-075a-437e-a7f9-bbcf0d220feb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:33:13.698481 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698461    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a94c7e93-075a-437e-a7f9-bbcf0d220feb" (UID: "a94c7e93-075a-437e-a7f9-bbcf0d220feb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:33:13.698749 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698525    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a94c7e93-075a-437e-a7f9-bbcf0d220feb" (UID: "a94c7e93-075a-437e-a7f9-bbcf0d220feb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:33:13.698749 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698740    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:33:13.698849 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698759    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:33:13.698849 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698768    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a94c7e93-075a-437e-a7f9-bbcf0d220feb" (UID: "a94c7e93-075a-437e-a7f9-bbcf0d220feb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:33:13.698849 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.698774    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:33:13.699367 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.699335    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-data" (OuterVolumeSpecName: "istio-data") pod "a94c7e93-075a-437e-a7f9-bbcf0d220feb" (UID: "a94c7e93-075a-437e-a7f9-bbcf0d220feb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:33:13.700537 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.700508    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-kube-api-access-lghjq" (OuterVolumeSpecName: "kube-api-access-lghjq") pod "a94c7e93-075a-437e-a7f9-bbcf0d220feb" (UID: "a94c7e93-075a-437e-a7f9-bbcf0d220feb"). InnerVolumeSpecName "kube-api-access-lghjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:33:13.700705 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.700682    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-token" (OuterVolumeSpecName: "istio-token") pod "a94c7e93-075a-437e-a7f9-bbcf0d220feb" (UID: "a94c7e93-075a-437e-a7f9-bbcf0d220feb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:33:13.700946 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.700922    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a94c7e93-075a-437e-a7f9-bbcf0d220feb" (UID: "a94c7e93-075a-437e-a7f9-bbcf0d220feb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:33:13.701164 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.701147    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a94c7e93-075a-437e-a7f9-bbcf0d220feb" (UID: "a94c7e93-075a-437e-a7f9-bbcf0d220feb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:33:13.800064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.799956    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:33:13.800064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.799989    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lghjq\" (UniqueName: \"kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-kube-api-access-lghjq\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:33:13.800064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.800028    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:33:13.800064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.800039    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:33:13.800064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.800048    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:33:13.800064 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.800056    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94c7e93-075a-437e-a7f9-bbcf0d220feb-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:33:13.972437 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.972400    2561 generic.go:356] "Generic (PLEG): container finished" podID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerID="98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53" exitCode=0
Jul 17 14:33:13.972610 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.972449    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"
Jul 17 14:33:13.972610 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.972484    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" event={"ID":"a94c7e93-075a-437e-a7f9-bbcf0d220feb","Type":"ContainerDied","Data":"98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53"}
Jul 17 14:33:13.972610 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.972543    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw" event={"ID":"a94c7e93-075a-437e-a7f9-bbcf0d220feb","Type":"ContainerDied","Data":"90fb9892eb956c51ec74f359e0f83e9dcf29b91894f05e6e7a1e2290a614195f"}
Jul 17 14:33:13.972610 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.972576    2561 scope.go:117] "RemoveContainer" containerID="98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53"
Jul 17 14:33:13.983205 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.983175    2561 scope.go:117] "RemoveContainer" containerID="03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d"
Jul 17 14:33:13.991952 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.991930    2561 scope.go:117] "RemoveContainer" containerID="98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53"
Jul 17 14:33:13.992328 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:33:13.992307    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53\": container with ID starting with 98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53 not found: ID does not exist" containerID="98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53"
Jul 17 14:33:13.992393 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.992337    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53"} err="failed to get container status \"98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53\": rpc error: code = NotFound desc = could not find container \"98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53\": container with ID starting with 98ba15553e77c1c5ec825354968e9015a30577dfacc4e6177a1981b4fc601d53 not found: ID does not exist"
Jul 17 14:33:13.992393 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.992355    2561 scope.go:117] "RemoveContainer" containerID="03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d"
Jul 17 14:33:13.992649 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:33:13.992635    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d\": container with ID starting with 03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d not found: ID does not exist" containerID="03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d"
Jul 17 14:33:13.992710 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:13.992652    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d"} err="failed to get container status \"03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d\": rpc error: code = NotFound desc = could not find container \"03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d\": container with ID starting with 03522cd8fdf3e849554beacade53ade2b9787f01fd2bf4847c3fe2804065e15d not found: ID does not exist"
Jul 17 14:33:14.004114 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:14.004085    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"]
Jul 17 14:33:14.008374 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:14.008351    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f387e370-openshift-default-947948c7f-dswmw"]
Jul 17 14:33:14.773923 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:33:14.773892    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" path="/var/lib/kubelet/pods/a94c7e93-075a-437e-a7f9-bbcf0d220feb/volumes"
Jul 17 14:34:05.769415 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.769381    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"]
Jul 17 14:34:05.769876 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.769794    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy"
Jul 17 14:34:05.769876 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.769807    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy"
Jul 17 14:34:05.769876 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.769816    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy"
Jul 17 14:34:05.769876 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.769821    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy"
Jul 17 14:34:05.769876 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.769828    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76ed735f-9820-48aa-ac4d-4819d2e1b8fb" containerName="main"
Jul 17 14:34:05.769876 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.769834    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="76ed735f-9820-48aa-ac4d-4819d2e1b8fb" containerName="main"
Jul 17 14:34:05.770147 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.769899    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy"
Jul 17 14:34:05.770147 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.769907    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="76ed735f-9820-48aa-ac4d-4819d2e1b8fb" containerName="main"
Jul 17 14:34:05.773079 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.773063    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.776930 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.776911    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-be4c79-openshift-default-dockercfg-p6cxc\""
Jul 17 14:34:05.791734 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.791715    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"]
Jul 17 14:34:05.886963 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.886914    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-data\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.887181 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.886973    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-certs\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.887181 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.887026    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njmcs\" (UniqueName: \"kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-kube-api-access-njmcs\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.887181 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.887076    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-socket\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.887181 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.887162    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-credential-socket\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.887385 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.887232    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-token\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.887385 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.887275    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-podinfo\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.887454 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.887373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istiod-ca-cert\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.887454 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.887410    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-envoy\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.988455 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.988430    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-credential-socket\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.988628 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.988470    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-token\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.988628 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.988495    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-podinfo\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.988755 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.988657    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istiod-ca-cert\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.988755 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.988693    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-envoy\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.988755 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.988747    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-data\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.988910 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.988777    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-certs\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.988910 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.988819    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njmcs\" (UniqueName: \"kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-kube-api-access-njmcs\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.988910 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.988834    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-credential-socket\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.989113 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.988981    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-socket\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.989173 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.989120    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-data\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.989250 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.989231    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-certs\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.989293 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.989278    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-socket\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.989401 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.989378    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istiod-ca-cert\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.991144 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.991122    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-envoy\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:05.991287 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:05.991270    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-podinfo\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:06.000820 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:06.000800    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-token\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:06.001916 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:06.001899    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njmcs\" (UniqueName: \"kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-kube-api-access-njmcs\") pod \"e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") " pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:06.083913 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:06.083826    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:06.234948 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:06.234923    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"]
Jul 17 14:34:06.236783 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:34:06.236749    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85f09bf1_1ac8_43c4_9763_ec8830f5260d.slice/crio-79e9f292021b44d53bb4a34e9cb1a9c8db4fea60bbf9d132979e4a221849fa4a WatchSource:0}: Error finding container 79e9f292021b44d53bb4a34e9cb1a9c8db4fea60bbf9d132979e4a221849fa4a: Status 404 returned error can't find the container with id 79e9f292021b44d53bb4a34e9cb1a9c8db4fea60bbf9d132979e4a221849fa4a
Jul 17 14:34:06.239041 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:06.238988    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:06.239144 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:06.239073    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:06.239144 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:06.239109    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:06.264684 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:06.264662    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" event={"ID":"85f09bf1-1ac8-43c4-9763-ec8830f5260d","Type":"ContainerStarted","Data":"79e9f292021b44d53bb4a34e9cb1a9c8db4fea60bbf9d132979e4a221849fa4a"}
Jul 17 14:34:07.271251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:07.271214    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" event={"ID":"85f09bf1-1ac8-43c4-9763-ec8830f5260d","Type":"ContainerStarted","Data":"19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9"}
Jul 17 14:34:07.297451 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:07.297403    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" podStartSLOduration=2.2973897 podStartE2EDuration="2.2973897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:34:07.294623096 +0000 UTC m=+2933.001069929" watchObservedRunningTime="2026-07-17 14:34:07.2973897 +0000 UTC m=+2933.003836533"
Jul 17 14:34:08.084834 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:08.084805    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:08.089662 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:08.089640    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:08.276769 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:08.276742    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:08.277765 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:08.277746    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:14.397384 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.397353    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"]
Jul 17 14:34:14.397860 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.397847    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94c7e93-075a-437e-a7f9-bbcf0d220feb" containerName="istio-proxy"
Jul 17 14:34:14.401207 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.401185    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.405660 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.405453    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-b9f570-openshift-default-dockercfg-99q2k\""
Jul 17 14:34:14.416155 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.416133    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"]
Jul 17 14:34:14.468887 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.468861    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbcaadda-e476-451d-a06a-03e2c5646450-istiod-ca-cert\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.469065 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.468900    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn6x6\" (UniqueName: \"kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-kube-api-access-qn6x6\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.469065 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.468929    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-socket\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.469065 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.469035    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-certs\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.469204 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.469067    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbcaadda-e476-451d-a06a-03e2c5646450-istio-podinfo\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.469204 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.469093    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-data\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.469204 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.469123    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-istio-token\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.469314 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.469242    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-envoy\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.469366 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.469339    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-credential-socket\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.570557 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.570525    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-credential-socket\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.570765 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.570575    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbcaadda-e476-451d-a06a-03e2c5646450-istiod-ca-cert\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.570765 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.570598    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn6x6\" (UniqueName: \"kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-kube-api-access-qn6x6\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.570765 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.570617    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-socket\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.570765 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.570656    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-certs\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.570765 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.570676    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbcaadda-e476-451d-a06a-03e2c5646450-istio-podinfo\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.570765 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.570693    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-data\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.570765 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.570729    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-istio-token\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.571216 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.570833    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-envoy\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.571216 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.571104    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-data\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.571339 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.571209    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-credential-socket\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.571339 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.571241    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-certs\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.571339 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.571311    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-socket\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.571487 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.571426    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbcaadda-e476-451d-a06a-03e2c5646450-istiod-ca-cert\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.573320 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.573291    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-envoy\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.573667 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.573649    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbcaadda-e476-451d-a06a-03e2c5646450-istio-podinfo\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.579646 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.579622    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn6x6\" (UniqueName: \"kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-kube-api-access-qn6x6\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.579851 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.579833    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-istio-token\") pod \"e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") " pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.715625 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.715599    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-b9f570-openshift-default-dockercfg-99q2k\""
Jul 17 14:34:14.723070 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.723044    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:14.879971 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.879942    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"]
Jul 17 14:34:14.881950 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:34:14.881916    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbcaadda_e476_451d_a06a_03e2c5646450.slice/crio-6455ec4337ff9e8ffe7da0cc473776943acd26e019e4fd7823d4554516095adb WatchSource:0}: Error finding container 6455ec4337ff9e8ffe7da0cc473776943acd26e019e4fd7823d4554516095adb: Status 404 returned error can't find the container with id 6455ec4337ff9e8ffe7da0cc473776943acd26e019e4fd7823d4554516095adb
Jul 17 14:34:14.884258 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.884224    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:14.884351 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.884288    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:14.884351 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:14.884340    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:15.315199 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:15.315168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" event={"ID":"fbcaadda-e476-451d-a06a-03e2c5646450","Type":"ContainerStarted","Data":"6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba"}
Jul 17 14:34:15.315384 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:15.315225    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" event={"ID":"fbcaadda-e476-451d-a06a-03e2c5646450","Type":"ContainerStarted","Data":"6455ec4337ff9e8ffe7da0cc473776943acd26e019e4fd7823d4554516095adb"}
Jul 17 14:34:15.724269 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:15.724227    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:15.726036 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:15.725967    2561 patch_prober.go:28] interesting pod/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Jul 17 14:34:15.726192 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:15.726051    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Jul 17 14:34:16.739631 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:16.739581    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:16.769736 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:16.769688    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" podStartSLOduration=2.76967295 podStartE2EDuration="2.76967295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:34:15.362410976 +0000 UTC m=+2941.068857811" watchObservedRunningTime="2026-07-17 14:34:16.76967295 +0000 UTC m=+2942.476119783"
Jul 17 14:34:17.329604 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:17.329567    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:17.343026 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:17.342292    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:23.277872 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:23.277834    2561 patch_prober.go:28] interesting pod/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Jul 17 14:34:23.278337 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:23.277913    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Jul 17 14:34:23.380601 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:23.380562    2561 generic.go:356] "Generic (PLEG): container finished" podID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerID="19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9" exitCode=0
Jul 17 14:34:23.380768 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:23.380634    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" event={"ID":"85f09bf1-1ac8-43c4-9763-ec8830f5260d","Type":"ContainerDied","Data":"19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9"}
Jul 17 14:34:23.381098 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:23.381084    2561 scope.go:117] "RemoveContainer" containerID="19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9"
Jul 17 14:34:23.382209 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:23.382183    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:23.382288 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:23.382269    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:23.382335 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:23.382303    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:24.387714 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:24.387677    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" event={"ID":"85f09bf1-1ac8-43c4-9763-ec8830f5260d","Type":"ContainerStarted","Data":"899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9"}
Jul 17 14:34:25.084713 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:25.084583    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:25.084973 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:25.084951    2561 patch_prober.go:28] interesting pod/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Jul 17 14:34:25.085040 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:25.085018    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Jul 17 14:34:26.084991 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:26.084951    2561 patch_prober.go:28] interesting pod/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Jul 17 14:34:26.085458 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:26.085209    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Jul 17 14:34:27.085143 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:27.085111    2561 patch_prober.go:28] interesting pod/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Jul 17 14:34:27.085499 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:27.085166    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Jul 17 14:34:28.084377 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:28.084342    2561 patch_prober.go:28] interesting pod/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Jul 17 14:34:28.084574 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:28.084405    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Jul 17 14:34:29.087912 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:29.087882    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:29.088326 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:29.088283    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:29.089054 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:29.089037    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:34:30.424777 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:30.424746    2561 generic.go:356] "Generic (PLEG): container finished" podID="fbcaadda-e476-451d-a06a-03e2c5646450" containerID="6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba" exitCode=0
Jul 17 14:34:30.425242 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:30.424788    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" event={"ID":"fbcaadda-e476-451d-a06a-03e2c5646450","Type":"ContainerDied","Data":"6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba"}
Jul 17 14:34:30.425405 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:30.425386    2561 scope.go:117] "RemoveContainer" containerID="6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba"
Jul 17 14:34:30.426525 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:30.426497    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:30.426628 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:30.426583    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:30.426692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:30.426660    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:34:31.436197 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:31.436157    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" event={"ID":"fbcaadda-e476-451d-a06a-03e2c5646450","Type":"ContainerStarted","Data":"2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b"}
Jul 17 14:34:31.725065 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:31.724948    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:31.725403 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:31.725380    2561 patch_prober.go:28] interesting pod/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Jul 17 14:34:31.725551 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:31.725528    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Jul 17 14:34:32.330048 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:32.329979    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:32.723956 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:32.723916    2561 patch_prober.go:28] interesting pod/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Jul 17 14:34:32.724474 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:32.723984    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Jul 17 14:34:33.724165 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:33.724122    2561 patch_prober.go:28] interesting pod/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Jul 17 14:34:33.724670 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:33.724214    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Jul 17 14:34:34.743145 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:34.743102    2561 patch_prober.go:28] interesting pod/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Jul 17 14:34:34.743643 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:34.743180    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Jul 17 14:34:35.727999 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:35.727911    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:34:35.729166 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:34:35.729143    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:35:07.892549 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:07.892513    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"]
Jul 17 14:35:07.896542 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:07.896521    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:07.900159 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:07.900134    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b9f570\"/\"default-dockercfg-pbwhj\""
Jul 17 14:35:07.900306 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:07.900205    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b9f570\"/\"test-model-b9f570-kserve-self-signed-certs\""
Jul 17 14:35:07.900306 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:07.900236    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b9f570\"/\"openshift-service-ca.crt\""
Jul 17 14:35:07.900306 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:07.900225    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b9f570\"/\"kube-root-ca.crt\""
Jul 17 14:35:07.912037 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:07.911991    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"]
Jul 17 14:35:08.000879 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.000844    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-home\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.000879 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.000879    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-tmp-dir\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.001083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.000899    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03b179f-2d7a-40c2-9667-d50376b0c095-tls-certs\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.001083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.001032    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-dshm\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.001083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.001072    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-model-cache\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.001188 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.001114    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgltk\" (UniqueName: \"kubernetes.io/projected/e03b179f-2d7a-40c2-9667-d50376b0c095-kube-api-access-sgltk\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.102160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.102129    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-home\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.102160 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.102161    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-tmp-dir\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.102363 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.102180    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03b179f-2d7a-40c2-9667-d50376b0c095-tls-certs\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.102363 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.102249    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-dshm\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.102363 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.102282    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-model-cache\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.102363 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.102319    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgltk\" (UniqueName: \"kubernetes.io/projected/e03b179f-2d7a-40c2-9667-d50376b0c095-kube-api-access-sgltk\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.102574 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.102519    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-home\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.102638 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.102588    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-tmp-dir\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.102638 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.102625    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-model-cache\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.104512 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.104490    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-dshm\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.104770 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.104752    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03b179f-2d7a-40c2-9667-d50376b0c095-tls-certs\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.111531 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.111512    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgltk\" (UniqueName: \"kubernetes.io/projected/e03b179f-2d7a-40c2-9667-d50376b0c095-kube-api-access-sgltk\") pod \"test-model-b9f570-kserve-84f5f4d587-b7t88\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") " pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.206514 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.206488    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:08.353333 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.353305    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"]
Jul 17 14:35:08.355476 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:35:08.355446    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode03b179f_2d7a_40c2_9667_d50376b0c095.slice/crio-b9c8617b43aa3e08617cb5488c263cd2e38ae978e1ae77ac011528a6ce6c8458 WatchSource:0}: Error finding container b9c8617b43aa3e08617cb5488c263cd2e38ae978e1ae77ac011528a6ce6c8458: Status 404 returned error can't find the container with id b9c8617b43aa3e08617cb5488c263cd2e38ae978e1ae77ac011528a6ce6c8458
Jul 17 14:35:08.657975 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.657886    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88" event={"ID":"e03b179f-2d7a-40c2-9667-d50376b0c095","Type":"ContainerStarted","Data":"c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7"}
Jul 17 14:35:08.657975 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.657928    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88" event={"ID":"e03b179f-2d7a-40c2-9667-d50376b0c095","Type":"ContainerStarted","Data":"b9c8617b43aa3e08617cb5488c263cd2e38ae978e1ae77ac011528a6ce6c8458"}
Jul 17 14:35:08.679893 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:08.679850    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88" podStartSLOduration=1.679835205 podStartE2EDuration="1.679835205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:35:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:35:08.678809395 +0000 UTC m=+2994.385256249" watchObservedRunningTime="2026-07-17 14:35:08.679835205 +0000 UTC m=+2994.386282039"
Jul 17 14:35:18.206759 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:18.206722    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:18.206759 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:18.206768    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:18.219240 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:18.219217    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:18.725305 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:18.725279    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:31.865411 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:31.865378    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"]
Jul 17 14:35:46.886053 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:46.885996    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88" podUID="e03b179f-2d7a-40c2-9667-d50376b0c095" containerName="main" containerID="cri-o://c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7" gracePeriod=45
Jul 17 14:35:47.149486 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.149426    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:47.280859 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.280829    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-model-cache\") pod \"e03b179f-2d7a-40c2-9667-d50376b0c095\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") "
Jul 17 14:35:47.280859 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.280862    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-tmp-dir\") pod \"e03b179f-2d7a-40c2-9667-d50376b0c095\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") "
Jul 17 14:35:47.281083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.280943    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgltk\" (UniqueName: \"kubernetes.io/projected/e03b179f-2d7a-40c2-9667-d50376b0c095-kube-api-access-sgltk\") pod \"e03b179f-2d7a-40c2-9667-d50376b0c095\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") "
Jul 17 14:35:47.281083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.280981    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-home\") pod \"e03b179f-2d7a-40c2-9667-d50376b0c095\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") "
Jul 17 14:35:47.281083 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.281029    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03b179f-2d7a-40c2-9667-d50376b0c095-tls-certs\") pod \"e03b179f-2d7a-40c2-9667-d50376b0c095\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") "
Jul 17 14:35:47.281226 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.281089    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-dshm\") pod \"e03b179f-2d7a-40c2-9667-d50376b0c095\" (UID: \"e03b179f-2d7a-40c2-9667-d50376b0c095\") "
Jul 17 14:35:47.281226 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.281130    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-model-cache" (OuterVolumeSpecName: "model-cache") pod "e03b179f-2d7a-40c2-9667-d50376b0c095" (UID: "e03b179f-2d7a-40c2-9667-d50376b0c095"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:35:47.281339 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.281237    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-home" (OuterVolumeSpecName: "home") pod "e03b179f-2d7a-40c2-9667-d50376b0c095" (UID: "e03b179f-2d7a-40c2-9667-d50376b0c095"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:35:47.281339 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.281243    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e03b179f-2d7a-40c2-9667-d50376b0c095" (UID: "e03b179f-2d7a-40c2-9667-d50376b0c095"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:35:47.281446 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.281418    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-home\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:47.281446 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.281431    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-model-cache\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:47.281521 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.281454    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-tmp-dir\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:47.283333 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.283302    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-dshm" (OuterVolumeSpecName: "dshm") pod "e03b179f-2d7a-40c2-9667-d50376b0c095" (UID: "e03b179f-2d7a-40c2-9667-d50376b0c095"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:35:47.283443 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.283377    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e03b179f-2d7a-40c2-9667-d50376b0c095-kube-api-access-sgltk" (OuterVolumeSpecName: "kube-api-access-sgltk") pod "e03b179f-2d7a-40c2-9667-d50376b0c095" (UID: "e03b179f-2d7a-40c2-9667-d50376b0c095"). InnerVolumeSpecName "kube-api-access-sgltk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:35:47.283443 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.283373    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e03b179f-2d7a-40c2-9667-d50376b0c095-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e03b179f-2d7a-40c2-9667-d50376b0c095" (UID: "e03b179f-2d7a-40c2-9667-d50376b0c095"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 14:35:47.382569 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.382542    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgltk\" (UniqueName: \"kubernetes.io/projected/e03b179f-2d7a-40c2-9667-d50376b0c095-kube-api-access-sgltk\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:47.382569 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.382567    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03b179f-2d7a-40c2-9667-d50376b0c095-tls-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:47.382569 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.382576    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03b179f-2d7a-40c2-9667-d50376b0c095-dshm\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:47.875750 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.875714    2561 generic.go:356] "Generic (PLEG): container finished" podID="e03b179f-2d7a-40c2-9667-d50376b0c095" containerID="c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7" exitCode=0
Jul 17 14:35:47.875914 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.875757    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"
Jul 17 14:35:47.875914 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.875799    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88" event={"ID":"e03b179f-2d7a-40c2-9667-d50376b0c095","Type":"ContainerDied","Data":"c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7"}
Jul 17 14:35:47.875914 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.875841    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88" event={"ID":"e03b179f-2d7a-40c2-9667-d50376b0c095","Type":"ContainerDied","Data":"b9c8617b43aa3e08617cb5488c263cd2e38ae978e1ae77ac011528a6ce6c8458"}
Jul 17 14:35:47.875914 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.875858    2561 scope.go:117] "RemoveContainer" containerID="c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7"
Jul 17 14:35:47.885762 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.885743    2561 scope.go:117] "RemoveContainer" containerID="c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7"
Jul 17 14:35:47.886051 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:35:47.886034    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7\": container with ID starting with c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7 not found: ID does not exist" containerID="c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7"
Jul 17 14:35:47.886128 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.886058    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7"} err="failed to get container status \"c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7\": rpc error: code = NotFound desc = could not find container \"c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7\": container with ID starting with c3517bba77129b117cebf77db39ebd1050690b8c1def67d0d4e58e77e18d55c7 not found: ID does not exist"
Jul 17 14:35:47.899610 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.899584    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"]
Jul 17 14:35:47.905579 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:47.905560    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-b9f570/test-model-b9f570-kserve-84f5f4d587-b7t88"]
Jul 17 14:35:48.775845 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:48.775815    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e03b179f-2d7a-40c2-9667-d50376b0c095" path="/var/lib/kubelet/pods/e03b179f-2d7a-40c2-9667-d50376b0c095/volumes"
Jul 17 14:35:54.230186 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:54.230153    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"]
Jul 17 14:35:54.230569 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:54.230386    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy" containerID="cri-o://899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9" gracePeriod=30
Jul 17 14:35:59.090886 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.090851    2561 patch_prober.go:28] interesting pod/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 14:35:59.091266 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.090906    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 14:35:59.503464 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.503439    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:35:59.599457 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599389    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-podinfo\") pod \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") "
Jul 17 14:35:59.599457 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599436    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-envoy\") pod \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") "
Jul 17 14:35:59.599457 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599453    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-socket\") pod \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") "
Jul 17 14:35:59.599725 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599514    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istiod-ca-cert\") pod \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") "
Jul 17 14:35:59.599725 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599550    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-data\") pod \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") "
Jul 17 14:35:59.599725 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599575    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-token\") pod \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") "
Jul 17 14:35:59.599725 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599612    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njmcs\" (UniqueName: \"kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-kube-api-access-njmcs\") pod \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") "
Jul 17 14:35:59.599725 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599643    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-credential-socket\") pod \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") "
Jul 17 14:35:59.599725 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599675    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-certs\") pod \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\" (UID: \"85f09bf1-1ac8-43c4-9763-ec8830f5260d\") "
Jul 17 14:35:59.600065 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599773    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85f09bf1-1ac8-43c4-9763-ec8830f5260d" (UID: "85f09bf1-1ac8-43c4-9763-ec8830f5260d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:35:59.600065 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.599917    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85f09bf1-1ac8-43c4-9763-ec8830f5260d" (UID: "85f09bf1-1ac8-43c4-9763-ec8830f5260d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:35:59.600346 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.600144    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85f09bf1-1ac8-43c4-9763-ec8830f5260d" (UID: "85f09bf1-1ac8-43c4-9763-ec8830f5260d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:35:59.600346 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.600177    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85f09bf1-1ac8-43c4-9763-ec8830f5260d" (UID: "85f09bf1-1ac8-43c4-9763-ec8830f5260d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:35:59.600346 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.600218    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:59.600346 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.600244    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:59.600570 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.600490    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-data" (OuterVolumeSpecName: "istio-data") pod "85f09bf1-1ac8-43c4-9763-ec8830f5260d" (UID: "85f09bf1-1ac8-43c4-9763-ec8830f5260d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:35:59.601829 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.601799    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85f09bf1-1ac8-43c4-9763-ec8830f5260d" (UID: "85f09bf1-1ac8-43c4-9763-ec8830f5260d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:35:59.601952 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.601832    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-token" (OuterVolumeSpecName: "istio-token") pod "85f09bf1-1ac8-43c4-9763-ec8830f5260d" (UID: "85f09bf1-1ac8-43c4-9763-ec8830f5260d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:35:59.601952 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.601856    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85f09bf1-1ac8-43c4-9763-ec8830f5260d" (UID: "85f09bf1-1ac8-43c4-9763-ec8830f5260d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:35:59.602113 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.601987    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-kube-api-access-njmcs" (OuterVolumeSpecName: "kube-api-access-njmcs") pod "85f09bf1-1ac8-43c4-9763-ec8830f5260d" (UID: "85f09bf1-1ac8-43c4-9763-ec8830f5260d"). InnerVolumeSpecName "kube-api-access-njmcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:35:59.701439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.701408    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:59.701439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.701432    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:59.701439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.701445    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njmcs\" (UniqueName: \"kubernetes.io/projected/85f09bf1-1ac8-43c4-9763-ec8830f5260d-kube-api-access-njmcs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:59.701637 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.701457    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:59.701637 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.701470    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:59.701637 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.701482    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:59.701637 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.701493    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85f09bf1-1ac8-43c4-9763-ec8830f5260d-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:35:59.951668 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.951638    2561 generic.go:356] "Generic (PLEG): container finished" podID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerID="899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9" exitCode=0
Jul 17 14:35:59.951816 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.951685    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"
Jul 17 14:35:59.951816 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.951724    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" event={"ID":"85f09bf1-1ac8-43c4-9763-ec8830f5260d","Type":"ContainerDied","Data":"899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9"}
Jul 17 14:35:59.951816 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.951773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4" event={"ID":"85f09bf1-1ac8-43c4-9763-ec8830f5260d","Type":"ContainerDied","Data":"79e9f292021b44d53bb4a34e9cb1a9c8db4fea60bbf9d132979e4a221849fa4a"}
Jul 17 14:35:59.951816 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.951794    2561 scope.go:117] "RemoveContainer" containerID="899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9"
Jul 17 14:35:59.961871 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.961858    2561 scope.go:117] "RemoveContainer" containerID="19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9"
Jul 17 14:35:59.970415 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.970397    2561 scope.go:117] "RemoveContainer" containerID="899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9"
Jul 17 14:35:59.970651 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:35:59.970633    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9\": container with ID starting with 899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9 not found: ID does not exist" containerID="899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9"
Jul 17 14:35:59.970699 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.970660    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9"} err="failed to get container status \"899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9\": rpc error: code = NotFound desc = could not find container \"899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9\": container with ID starting with 899d74ca81e6eda5331662c6350132fe6bfcbf495161f7d9576caf17c8dd32a9 not found: ID does not exist"
Jul 17 14:35:59.970699 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.970677    2561 scope.go:117] "RemoveContainer" containerID="19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9"
Jul 17 14:35:59.970884 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:35:59.970870    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9\": container with ID starting with 19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9 not found: ID does not exist" containerID="19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9"
Jul 17 14:35:59.970925 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.970889    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9"} err="failed to get container status \"19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9\": rpc error: code = NotFound desc = could not find container \"19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9\": container with ID starting with 19254bb7e6b1a5b6c19e05efb14bef0d724772457c83dbfaa36512fe23d46cf9 not found: ID does not exist"
Jul 17 14:35:59.975976 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.975953    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"]
Jul 17 14:35:59.983237 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:35:59.983215    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-be4c79-openshift-default-5cc5d9c78f-n8kz4"]
Jul 17 14:36:00.772439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:00.772410    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" path="/var/lib/kubelet/pods/85f09bf1-1ac8-43c4-9763-ec8830f5260d/volumes"
Jul 17 14:36:16.651911 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:16.651875    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"]
Jul 17 14:36:16.652377 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:16.652108    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy" containerID="cri-o://2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b" gracePeriod=30
Jul 17 14:36:17.599112 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.599077    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"]
Jul 17 14:36:17.599506 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.599493    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e03b179f-2d7a-40c2-9667-d50376b0c095" containerName="main"
Jul 17 14:36:17.599550 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.599507    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e03b179f-2d7a-40c2-9667-d50376b0c095" containerName="main"
Jul 17 14:36:17.599550 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.599526    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy"
Jul 17 14:36:17.599550 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.599532    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy"
Jul 17 14:36:17.599550 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.599548    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy"
Jul 17 14:36:17.599673 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.599553    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy"
Jul 17 14:36:17.599673 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.599616    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy"
Jul 17 14:36:17.599673 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.599626    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e03b179f-2d7a-40c2-9667-d50376b0c095" containerName="main"
Jul 17 14:36:17.604382 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.604362    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.607398 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.607376    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8b734e-openshift-default-dockercfg-g9jjp\""
Jul 17 14:36:17.622697 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.622672    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"]
Jul 17 14:36:17.665155 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.665127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-certs\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.665520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.665183    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-data\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.665520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.665240    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-socket\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.665520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.665293    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-podinfo\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.665520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.665326    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9lrv\" (UniqueName: \"kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-kube-api-access-s9lrv\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.665520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.665362    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-envoy\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.665520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.665384    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-credential-socket\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.665520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.665399    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6de929b3-0d80-450f-9875-0fb8ee8fff63-istiod-ca-cert\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.665520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.665423    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-token\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766192 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766163    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-certs\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766493 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766210    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-data\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766493 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-socket\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766493 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766398    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-podinfo\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766493 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766465    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9lrv\" (UniqueName: \"kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-kube-api-access-s9lrv\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766734 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766503    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-envoy\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766734 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766534    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-credential-socket\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766734 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766589    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-data\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766734 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766632    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-certs\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766734 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766681    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-socket\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766999 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766842    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-credential-socket\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766999 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766885    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6de929b3-0d80-450f-9875-0fb8ee8fff63-istiod-ca-cert\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.766999 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.766928    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-token\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.767297 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.767279    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6de929b3-0d80-450f-9875-0fb8ee8fff63-istiod-ca-cert\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.769353 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.769330    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-envoy\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.769455 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.769440    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-podinfo\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.777337 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.777311    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9lrv\" (UniqueName: \"kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-kube-api-access-s9lrv\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.779393 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.779374    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-token\") pod \"e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") " pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:17.914876 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:17.914794    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:18.060112 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:18.060085    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"]
Jul 17 14:36:18.062172 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:36:18.062140    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6de929b3_0d80_450f_9875_0fb8ee8fff63.slice/crio-f686a0249796fc7c8e5c4c2d919f962fcac87c3bed0625745f71308c72e84cff WatchSource:0}: Error finding container f686a0249796fc7c8e5c4c2d919f962fcac87c3bed0625745f71308c72e84cff: Status 404 returned error can't find the container with id f686a0249796fc7c8e5c4c2d919f962fcac87c3bed0625745f71308c72e84cff
Jul 17 14:36:18.064255 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:18.064224    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:18.064364 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:18.064304    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:18.064364 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:18.064349    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:19.058921 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:19.058881    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" event={"ID":"6de929b3-0d80-450f-9875-0fb8ee8fff63","Type":"ContainerStarted","Data":"1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830"}
Jul 17 14:36:19.059380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:19.058944    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" event={"ID":"6de929b3-0d80-450f-9875-0fb8ee8fff63","Type":"ContainerStarted","Data":"f686a0249796fc7c8e5c4c2d919f962fcac87c3bed0625745f71308c72e84cff"}
Jul 17 14:36:19.088574 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:19.088528    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" podStartSLOduration=2.088514455 podStartE2EDuration="2.088514455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:36:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:36:19.085945214 +0000 UTC m=+3064.792392049" watchObservedRunningTime="2026-07-17 14:36:19.088514455 +0000 UTC m=+3064.794961288"
Jul 17 14:36:19.915605 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:19.915570    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:19.920719 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:19.920695    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:20.064464 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:20.064432    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:20.065304 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:20.065288    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:20.730508 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:20.730469    2561 patch_prober.go:28] interesting pod/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 14:36:20.730707 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:20.730528    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 14:36:21.917407 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:21.917389    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:36:22.018534 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.018504    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbcaadda-e476-451d-a06a-03e2c5646450-istiod-ca-cert\") pod \"fbcaadda-e476-451d-a06a-03e2c5646450\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") "
Jul 17 14:36:22.018692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.018561    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbcaadda-e476-451d-a06a-03e2c5646450-istio-podinfo\") pod \"fbcaadda-e476-451d-a06a-03e2c5646450\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") "
Jul 17 14:36:22.018692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.018584    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn6x6\" (UniqueName: \"kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-kube-api-access-qn6x6\") pod \"fbcaadda-e476-451d-a06a-03e2c5646450\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") "
Jul 17 14:36:22.018827 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.018724    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-envoy\") pod \"fbcaadda-e476-451d-a06a-03e2c5646450\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") "
Jul 17 14:36:22.018827 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.018771    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-credential-socket\") pod \"fbcaadda-e476-451d-a06a-03e2c5646450\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") "
Jul 17 14:36:22.018827 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.018819    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-istio-token\") pod \"fbcaadda-e476-451d-a06a-03e2c5646450\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") "
Jul 17 14:36:22.018993 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.018871    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-socket\") pod \"fbcaadda-e476-451d-a06a-03e2c5646450\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") "
Jul 17 14:36:22.018993 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.018893    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbcaadda-e476-451d-a06a-03e2c5646450-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fbcaadda-e476-451d-a06a-03e2c5646450" (UID: "fbcaadda-e476-451d-a06a-03e2c5646450"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:36:22.018993 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.018920    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-certs\") pod \"fbcaadda-e476-451d-a06a-03e2c5646450\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") "
Jul 17 14:36:22.018993 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.018952    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-data\") pod \"fbcaadda-e476-451d-a06a-03e2c5646450\" (UID: \"fbcaadda-e476-451d-a06a-03e2c5646450\") "
Jul 17 14:36:22.019234 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.019112    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fbcaadda-e476-451d-a06a-03e2c5646450" (UID: "fbcaadda-e476-451d-a06a-03e2c5646450"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:36:22.019442 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.019345    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fbcaadda-e476-451d-a06a-03e2c5646450" (UID: "fbcaadda-e476-451d-a06a-03e2c5646450"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:36:22.019580 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.019470    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fbcaadda-e476-451d-a06a-03e2c5646450" (UID: "fbcaadda-e476-451d-a06a-03e2c5646450"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:36:22.019781 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.019756    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbcaadda-e476-451d-a06a-03e2c5646450-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:36:22.019781 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.019781    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:36:22.019941 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.019794    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:36:22.019941 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.019824    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:36:22.020094 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.019996    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-data" (OuterVolumeSpecName: "istio-data") pod "fbcaadda-e476-451d-a06a-03e2c5646450" (UID: "fbcaadda-e476-451d-a06a-03e2c5646450"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:36:22.020960 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.020939    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-istio-token" (OuterVolumeSpecName: "istio-token") pod "fbcaadda-e476-451d-a06a-03e2c5646450" (UID: "fbcaadda-e476-451d-a06a-03e2c5646450"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:36:22.021272 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.021183    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbcaadda-e476-451d-a06a-03e2c5646450-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fbcaadda-e476-451d-a06a-03e2c5646450" (UID: "fbcaadda-e476-451d-a06a-03e2c5646450"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:36:22.021490 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.021465    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fbcaadda-e476-451d-a06a-03e2c5646450" (UID: "fbcaadda-e476-451d-a06a-03e2c5646450"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:36:22.021571 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.021552    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-kube-api-access-qn6x6" (OuterVolumeSpecName: "kube-api-access-qn6x6") pod "fbcaadda-e476-451d-a06a-03e2c5646450" (UID: "fbcaadda-e476-451d-a06a-03e2c5646450"). InnerVolumeSpecName "kube-api-access-qn6x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:36:22.034530 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.034506    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"]
Jul 17 14:36:22.034921 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.034907    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy"
Jul 17 14:36:22.034921 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.034921    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy"
Jul 17 14:36:22.035075 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.035032    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy"
Jul 17 14:36:22.035075 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.035044    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="85f09bf1-1ac8-43c4-9763-ec8830f5260d" containerName="istio-proxy"
Jul 17 14:36:22.035075 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.035051    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy"
Jul 17 14:36:22.038488 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.038470    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"
Jul 17 14:36:22.042210 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.042192    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 14:36:22.042312 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.042192    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kfbc4\""
Jul 17 14:36:22.049528 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.049507    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"]
Jul 17 14:36:22.076393 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.076366    2561 generic.go:356] "Generic (PLEG): container finished" podID="fbcaadda-e476-451d-a06a-03e2c5646450" containerID="2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b" exitCode=0
Jul 17 14:36:22.076506 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.076410    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"
Jul 17 14:36:22.076506 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.076450    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" event={"ID":"fbcaadda-e476-451d-a06a-03e2c5646450","Type":"ContainerDied","Data":"2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b"}
Jul 17 14:36:22.076506 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.076496    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx" event={"ID":"fbcaadda-e476-451d-a06a-03e2c5646450","Type":"ContainerDied","Data":"6455ec4337ff9e8ffe7da0cc473776943acd26e019e4fd7823d4554516095adb"}
Jul 17 14:36:22.076656 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.076519    2561 scope.go:117] "RemoveContainer" containerID="2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b"
Jul 17 14:36:22.086270 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.086254    2561 scope.go:117] "RemoveContainer" containerID="6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba"
Jul 17 14:36:22.094682 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.094668    2561 scope.go:117] "RemoveContainer" containerID="2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b"
Jul 17 14:36:22.094929 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:36:22.094913    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b\": container with ID starting with 2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b not found: ID does not exist" containerID="2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b"
Jul 17 14:36:22.094970 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.094945    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b"} err="failed to get container status \"2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b\": rpc error: code = NotFound desc = could not find container \"2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b\": container with ID starting with 2800157370be169f8cdd57135d68c53371cc8c99c4fcde84c67df0508686094b not found: ID does not exist"
Jul 17 14:36:22.094970 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.094960    2561 scope.go:117] "RemoveContainer" containerID="6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba"
Jul 17 14:36:22.095193 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:36:22.095179    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba\": container with ID starting with 6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba not found: ID does not exist" containerID="6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba"
Jul 17 14:36:22.095232 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.095197    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba"} err="failed to get container status \"6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba\": rpc error: code = NotFound desc = could not find container \"6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba\": container with ID starting with 6d33f6d199182aa984c92344fdd34f6d146d45f217eddc239e1b58c52a8913ba not found: ID does not exist"
Jul 17 14:36:22.099422 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.099403    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"]
Jul 17 14:36:22.106125 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.106106    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-b9f570-openshift-default-5bc469d74f-wttvx"]
Jul 17 14:36:22.120352 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.120329    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e06a34-b66f-4380-b562-30a6c18a5614-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-gxfrp\" (UID: \"b9e06a34-b66f-4380-b562-30a6c18a5614\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"
Jul 17 14:36:22.120461 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.120450    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbcaadda-e476-451d-a06a-03e2c5646450-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:36:22.120520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.120463    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn6x6\" (UniqueName: \"kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-kube-api-access-qn6x6\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:36:22.120520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.120472    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:36:22.120520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.120480    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbcaadda-e476-451d-a06a-03e2c5646450-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:36:22.120520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.120487    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbcaadda-e476-451d-a06a-03e2c5646450-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:36:22.221032 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.220930    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e06a34-b66f-4380-b562-30a6c18a5614-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-gxfrp\" (UID: \"b9e06a34-b66f-4380-b562-30a6c18a5614\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"
Jul 17 14:36:22.221716 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.221688    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e06a34-b66f-4380-b562-30a6c18a5614-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7874222-gxfrp\" (UID: \"b9e06a34-b66f-4380-b562-30a6c18a5614\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"
Jul 17 14:36:22.349851 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.349822    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"
Jul 17 14:36:22.491117 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.491091    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"]
Jul 17 14:36:22.493086 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:36:22.493060    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e06a34_b66f_4380_b562_30a6c18a5614.slice/crio-50e1c672d1341d26b793a00c3364e4359bc1b5f11c0924a95e61fc9b69eca208 WatchSource:0}: Error finding container 50e1c672d1341d26b793a00c3364e4359bc1b5f11c0924a95e61fc9b69eca208: Status 404 returned error can't find the container with id 50e1c672d1341d26b793a00c3364e4359bc1b5f11c0924a95e61fc9b69eca208
Jul 17 14:36:22.772670 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:22.772638    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" path="/var/lib/kubelet/pods/fbcaadda-e476-451d-a06a-03e2c5646450/volumes"
Jul 17 14:36:23.084942 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:23.084860    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" event={"ID":"b9e06a34-b66f-4380-b562-30a6c18a5614","Type":"ContainerStarted","Data":"ff2b07bf1ff721ae045a74f5cb11748499b219b5e8d01760a540f9bfa56d070b"}
Jul 17 14:36:23.084942 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:23.084908    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" event={"ID":"b9e06a34-b66f-4380-b562-30a6c18a5614","Type":"ContainerStarted","Data":"50e1c672d1341d26b793a00c3364e4359bc1b5f11c0924a95e61fc9b69eca208"}
Jul 17 14:36:23.106320 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:23.106281    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" podStartSLOduration=1.106266455 podStartE2EDuration="1.106266455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:36:23.104314783 +0000 UTC m=+3068.810761637" watchObservedRunningTime="2026-07-17 14:36:23.106266455 +0000 UTC m=+3068.812713288"
Jul 17 14:36:24.092434 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:24.092397    2561 generic.go:356] "Generic (PLEG): container finished" podID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerID="ff2b07bf1ff721ae045a74f5cb11748499b219b5e8d01760a540f9bfa56d070b" exitCode=6
Jul 17 14:36:24.092810 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:24.092462    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" event={"ID":"b9e06a34-b66f-4380-b562-30a6c18a5614","Type":"ContainerDied","Data":"ff2b07bf1ff721ae045a74f5cb11748499b219b5e8d01760a540f9bfa56d070b"}
Jul 17 14:36:24.092810 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:24.092739    2561 scope.go:117] "RemoveContainer" containerID="ff2b07bf1ff721ae045a74f5cb11748499b219b5e8d01760a540f9bfa56d070b"
Jul 17 14:36:25.100304 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:25.100265    2561 generic.go:356] "Generic (PLEG): container finished" podID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerID="e02663d538097a5e7c5befcb096257fdbeb835af61c3a2740b30d64035ba6d02" exitCode=6
Jul 17 14:36:25.100721 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:25.100310    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" event={"ID":"b9e06a34-b66f-4380-b562-30a6c18a5614","Type":"ContainerDied","Data":"e02663d538097a5e7c5befcb096257fdbeb835af61c3a2740b30d64035ba6d02"}
Jul 17 14:36:25.100721 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:25.100378    2561 scope.go:117] "RemoveContainer" containerID="ff2b07bf1ff721ae045a74f5cb11748499b219b5e8d01760a540f9bfa56d070b"
Jul 17 14:36:25.100721 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:25.100550    2561 scope.go:117] "RemoveContainer" containerID="e02663d538097a5e7c5befcb096257fdbeb835af61c3a2740b30d64035ba6d02"
Jul 17 14:36:25.100909 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:36:25.100780    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-gxfrp_odh-ai-gateway-infra(b9e06a34-b66f-4380-b562-30a6c18a5614)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614"
Jul 17 14:36:26.106919 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.106887    2561 scope.go:117] "RemoveContainer" containerID="e02663d538097a5e7c5befcb096257fdbeb835af61c3a2740b30d64035ba6d02"
Jul 17 14:36:26.107324 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:36:26.107100    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-gxfrp_odh-ai-gateway-infra(b9e06a34-b66f-4380-b562-30a6c18a5614)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614"
Jul 17 14:36:26.346793 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.346754    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"]
Jul 17 14:36:26.347286 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.347271    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy"
Jul 17 14:36:26.347344 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.347291    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbcaadda-e476-451d-a06a-03e2c5646450" containerName="istio-proxy"
Jul 17 14:36:26.351720 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.351699    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.354845 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.354821    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b55149-openshift-default-dockercfg-8nqh6\""
Jul 17 14:36:26.383509 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.383482    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"]
Jul 17 14:36:26.465397 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.465364    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-podinfo\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.465610 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.465440    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-token\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.465610 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.465543    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istiod-ca-cert\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.465610 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.465584    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-credential-socket\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.465789 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.465625    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-envoy\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.465789 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.465691    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnqvt\" (UniqueName: \"kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-kube-api-access-tnqvt\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.465789 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.465752    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-certs\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.465939 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.465795    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-socket\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.465939 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.465824    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-data\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.566581 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.566541    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-credential-socket\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.566762 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.566606    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-envoy\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.566762 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.566653    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnqvt\" (UniqueName: \"kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-kube-api-access-tnqvt\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.566762 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.566687    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-certs\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.566762 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.566715    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-socket\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.566762 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.566743    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-data\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.567044 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.566772    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-podinfo\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.567044 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.566829    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-token\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.567044 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.566868    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istiod-ca-cert\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.567214 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.567038    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-credential-socket\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.567214 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.567092    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-socket\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.567214 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.567148    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-certs\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.567369 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.567278    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-data\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.567537 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.567488    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istiod-ca-cert\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.569168 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.569141    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-envoy\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.569404 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.569388    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-podinfo\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.579869 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.579827    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnqvt\" (UniqueName: \"kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-kube-api-access-tnqvt\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.581920 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.581886    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-token\") pod \"e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") " pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.665119 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.665060    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:26.826254 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.826221    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"]
Jul 17 14:36:26.827192 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:36:26.827166    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecd82ab1_9dc9_4fe9_a76e_b9773df20106.slice/crio-c294bd9b8d1af1374cdb3fca47fe8367d1d9063b567ce7bc7dd882d925f09e85 WatchSource:0}: Error finding container c294bd9b8d1af1374cdb3fca47fe8367d1d9063b567ce7bc7dd882d925f09e85: Status 404 returned error can't find the container with id c294bd9b8d1af1374cdb3fca47fe8367d1d9063b567ce7bc7dd882d925f09e85
Jul 17 14:36:26.829879 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.829845    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:26.829943 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.829924    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:26.829984 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:26.829968    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:27.114968 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:27.114923    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" event={"ID":"ecd82ab1-9dc9-4fe9-a76e-b9773df20106","Type":"ContainerStarted","Data":"9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567"}
Jul 17 14:36:27.115410 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:27.114986    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" event={"ID":"ecd82ab1-9dc9-4fe9-a76e-b9773df20106","Type":"ContainerStarted","Data":"c294bd9b8d1af1374cdb3fca47fe8367d1d9063b567ce7bc7dd882d925f09e85"}
Jul 17 14:36:27.147276 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:27.147221    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" podStartSLOduration=1.147205564 podStartE2EDuration="1.147205564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:36:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:36:27.145057732 +0000 UTC m=+3072.851504565" watchObservedRunningTime="2026-07-17 14:36:27.147205564 +0000 UTC m=+3072.853652398"
Jul 17 14:36:27.665573 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:27.665537    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:27.670972 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:27.670937    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:28.122564 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:28.122525    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:28.127089 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:28.127056    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:35.205676 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:35.205637    2561 patch_prober.go:28] interesting pod/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": read tcp 10.133.0.2:56694->10.133.0.98:15021: read: connection reset by peer" start-of-body=
Jul 17 14:36:35.206117 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:35.205713    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": read tcp 10.133.0.2:56694->10.133.0.98:15021: read: connection reset by peer"
Jul 17 14:36:36.183877 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:36.183847    2561 generic.go:356] "Generic (PLEG): container finished" podID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerID="1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830" exitCode=0
Jul 17 14:36:36.184085 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:36.183917    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" event={"ID":"6de929b3-0d80-450f-9875-0fb8ee8fff63","Type":"ContainerDied","Data":"1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830"}
Jul 17 14:36:36.184369 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:36.184355    2561 scope.go:117] "RemoveContainer" containerID="1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830"
Jul 17 14:36:36.185351 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:36.185325    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:36.185424 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:36.185392    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:36.185464 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:36.185426    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:37.191209 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:37.191176    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" event={"ID":"6de929b3-0d80-450f-9875-0fb8ee8fff63","Type":"ContainerStarted","Data":"4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7"}
Jul 17 14:36:37.914966 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:37.914935    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:37.915297 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:37.915272    2561 patch_prober.go:28] interesting pod/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Jul 17 14:36:37.915406 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:37.915332    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Jul 17 14:36:38.915228 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:38.915189    2561 patch_prober.go:28] interesting pod/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Jul 17 14:36:38.915699 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:38.915248    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Jul 17 14:36:39.915303 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:39.915273    2561 patch_prober.go:28] interesting pod/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Jul 17 14:36:39.915666 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:39.915352    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Jul 17 14:36:40.766665 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:40.766632    2561 scope.go:117] "RemoveContainer" containerID="e02663d538097a5e7c5befcb096257fdbeb835af61c3a2740b30d64035ba6d02"
Jul 17 14:36:40.915537 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:40.915505    2561 patch_prober.go:28] interesting pod/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Jul 17 14:36:40.915948 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:40.915562    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Jul 17 14:36:41.214552 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:41.214511    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" event={"ID":"b9e06a34-b66f-4380-b562-30a6c18a5614","Type":"ContainerStarted","Data":"d1fab5457c9b99d3516b4e21af2b8809237866925894afec1c96743b50676fee"}
Jul 17 14:36:41.919218 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:41.919181    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:41.919586 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:41.919568    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:41.920255 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:41.920234    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:36:42.222269 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.222223    2561 generic.go:356] "Generic (PLEG): container finished" podID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerID="d1fab5457c9b99d3516b4e21af2b8809237866925894afec1c96743b50676fee" exitCode=6
Jul 17 14:36:42.222452 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.222278    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" event={"ID":"b9e06a34-b66f-4380-b562-30a6c18a5614","Type":"ContainerDied","Data":"d1fab5457c9b99d3516b4e21af2b8809237866925894afec1c96743b50676fee"}
Jul 17 14:36:42.222452 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.222354    2561 scope.go:117] "RemoveContainer" containerID="e02663d538097a5e7c5befcb096257fdbeb835af61c3a2740b30d64035ba6d02"
Jul 17 14:36:42.222691 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.222674    2561 scope.go:117] "RemoveContainer" containerID="d1fab5457c9b99d3516b4e21af2b8809237866925894afec1c96743b50676fee"
Jul 17 14:36:42.222951 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:36:42.222923    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7874222-gxfrp_odh-ai-gateway-infra(b9e06a34-b66f-4380-b562-30a6c18a5614)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614"
Jul 17 14:36:42.224405 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.224382    2561 generic.go:356] "Generic (PLEG): container finished" podID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerID="9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567" exitCode=0
Jul 17 14:36:42.224508 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.224445    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" event={"ID":"ecd82ab1-9dc9-4fe9-a76e-b9773df20106","Type":"ContainerDied","Data":"9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567"}
Jul 17 14:36:42.224998 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.224981    2561 scope.go:117] "RemoveContainer" containerID="9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567"
Jul 17 14:36:42.226239 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.226202    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:42.226332 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.226291    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:42.226381 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.226338    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:36:42.253142 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:42.253119    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"]
Jul 17 14:36:43.123027 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:43.122974    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:43.246075 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:43.246037    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" event={"ID":"ecd82ab1-9dc9-4fe9-a76e-b9773df20106","Type":"ContainerStarted","Data":"357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646"}
Jul 17 14:36:43.407682 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:43.407658    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"
Jul 17 14:36:43.557632 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:43.557598    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e06a34-b66f-4380-b562-30a6c18a5614-maas-api-service-ca\") pod \"b9e06a34-b66f-4380-b562-30a6c18a5614\" (UID: \"b9e06a34-b66f-4380-b562-30a6c18a5614\") "
Jul 17 14:36:43.557943 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:43.557922    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9e06a34-b66f-4380-b562-30a6c18a5614-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b9e06a34-b66f-4380-b562-30a6c18a5614" (UID: "b9e06a34-b66f-4380-b562-30a6c18a5614"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:36:43.658517 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:43.658418    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e06a34-b66f-4380-b562-30a6c18a5614-maas-api-service-ca\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:36:43.666225 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:43.666198    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:43.667435 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:43.667414    2561 patch_prober.go:28] interesting pod/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Jul 17 14:36:43.667535 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:43.667462    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Jul 17 14:36:44.252179 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:44.252146    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"
Jul 17 14:36:44.252661 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:44.252182    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp" event={"ID":"b9e06a34-b66f-4380-b562-30a6c18a5614","Type":"ContainerDied","Data":"50e1c672d1341d26b793a00c3364e4359bc1b5f11c0924a95e61fc9b69eca208"}
Jul 17 14:36:44.252661 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:44.252252    2561 scope.go:117] "RemoveContainer" containerID="d1fab5457c9b99d3516b4e21af2b8809237866925894afec1c96743b50676fee"
Jul 17 14:36:44.278439 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:44.278412    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"]
Jul 17 14:36:44.286960 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:44.286933    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7874222-gxfrp"]
Jul 17 14:36:44.666050 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:44.665937    2561 patch_prober.go:28] interesting pod/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Jul 17 14:36:44.666050 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:44.665990    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Jul 17 14:36:44.772759 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:44.772730    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614" path="/var/lib/kubelet/pods/b9e06a34-b66f-4380-b562-30a6c18a5614/volumes"
Jul 17 14:36:45.665800 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:45.665760    2561 patch_prober.go:28] interesting pod/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Jul 17 14:36:45.666223 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:45.665818    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Jul 17 14:36:46.665635 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:46.665599    2561 patch_prober.go:28] interesting pod/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Jul 17 14:36:46.665819 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:46.665655    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Jul 17 14:36:47.669176 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:47.669144    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:47.669587 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:47.669434    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:36:47.670230 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:36:47.670206    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:37:08.782895 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:08.782858    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"]
Jul 17 14:37:08.783419 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:08.783086    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy" containerID="cri-o://4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7" gracePeriod=30
Jul 17 14:37:11.921566 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:11.921526    2561 patch_prober.go:28] interesting pod/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 14:37:11.922072 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:11.921610    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 14:37:14.043348 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.043291    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:37:14.149153 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149120    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-certs\") pod \"6de929b3-0d80-450f-9875-0fb8ee8fff63\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") "
Jul 17 14:37:14.149299 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149162    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9lrv\" (UniqueName: \"kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-kube-api-access-s9lrv\") pod \"6de929b3-0d80-450f-9875-0fb8ee8fff63\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") "
Jul 17 14:37:14.149299 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149191    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-envoy\") pod \"6de929b3-0d80-450f-9875-0fb8ee8fff63\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") "
Jul 17 14:37:14.149299 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149250    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-podinfo\") pod \"6de929b3-0d80-450f-9875-0fb8ee8fff63\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") "
Jul 17 14:37:14.149299 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149282    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-token\") pod \"6de929b3-0d80-450f-9875-0fb8ee8fff63\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") "
Jul 17 14:37:14.149530 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149324    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-socket\") pod \"6de929b3-0d80-450f-9875-0fb8ee8fff63\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") "
Jul 17 14:37:14.149530 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149358    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-credential-socket\") pod \"6de929b3-0d80-450f-9875-0fb8ee8fff63\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") "
Jul 17 14:37:14.149530 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149364    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6de929b3-0d80-450f-9875-0fb8ee8fff63" (UID: "6de929b3-0d80-450f-9875-0fb8ee8fff63"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:37:14.149530 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149416    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6de929b3-0d80-450f-9875-0fb8ee8fff63-istiod-ca-cert\") pod \"6de929b3-0d80-450f-9875-0fb8ee8fff63\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") "
Jul 17 14:37:14.149530 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149452    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-data\") pod \"6de929b3-0d80-450f-9875-0fb8ee8fff63\" (UID: \"6de929b3-0d80-450f-9875-0fb8ee8fff63\") "
Jul 17 14:37:14.149801 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149580    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6de929b3-0d80-450f-9875-0fb8ee8fff63" (UID: "6de929b3-0d80-450f-9875-0fb8ee8fff63"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:37:14.149866 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149847    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6de929b3-0d80-450f-9875-0fb8ee8fff63" (UID: "6de929b3-0d80-450f-9875-0fb8ee8fff63"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:37:14.150134 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149887    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:14.150134 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.150075    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:14.150134 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.149883    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6de929b3-0d80-450f-9875-0fb8ee8fff63-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6de929b3-0d80-450f-9875-0fb8ee8fff63" (UID: "6de929b3-0d80-450f-9875-0fb8ee8fff63"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:37:14.150565 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.150540    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-data" (OuterVolumeSpecName: "istio-data") pod "6de929b3-0d80-450f-9875-0fb8ee8fff63" (UID: "6de929b3-0d80-450f-9875-0fb8ee8fff63"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:37:14.151622 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.151593    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-kube-api-access-s9lrv" (OuterVolumeSpecName: "kube-api-access-s9lrv") pod "6de929b3-0d80-450f-9875-0fb8ee8fff63" (UID: "6de929b3-0d80-450f-9875-0fb8ee8fff63"). InnerVolumeSpecName "kube-api-access-s9lrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:37:14.151715 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.151642    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6de929b3-0d80-450f-9875-0fb8ee8fff63" (UID: "6de929b3-0d80-450f-9875-0fb8ee8fff63"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:37:14.151715 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.151676    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6de929b3-0d80-450f-9875-0fb8ee8fff63" (UID: "6de929b3-0d80-450f-9875-0fb8ee8fff63"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:37:14.151824 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.151808    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-token" (OuterVolumeSpecName: "istio-token") pod "6de929b3-0d80-450f-9875-0fb8ee8fff63" (UID: "6de929b3-0d80-450f-9875-0fb8ee8fff63"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:37:14.251482 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.251457    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:14.251482 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.251481    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9lrv\" (UniqueName: \"kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-kube-api-access-s9lrv\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:14.251612 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.251491    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:14.251612 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.251499    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:14.251612 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.251506    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6de929b3-0d80-450f-9875-0fb8ee8fff63-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:14.251612 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.251515    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6de929b3-0d80-450f-9875-0fb8ee8fff63-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:14.251612 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.251522    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6de929b3-0d80-450f-9875-0fb8ee8fff63-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:14.426271 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.426243    2561 generic.go:356] "Generic (PLEG): container finished" podID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerID="4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7" exitCode=0
Jul 17 14:37:14.426455 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.426287    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"
Jul 17 14:37:14.426455 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.426328    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" event={"ID":"6de929b3-0d80-450f-9875-0fb8ee8fff63","Type":"ContainerDied","Data":"4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7"}
Jul 17 14:37:14.426455 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.426389    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j" event={"ID":"6de929b3-0d80-450f-9875-0fb8ee8fff63","Type":"ContainerDied","Data":"f686a0249796fc7c8e5c4c2d919f962fcac87c3bed0625745f71308c72e84cff"}
Jul 17 14:37:14.426455 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.426423    2561 scope.go:117] "RemoveContainer" containerID="4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7"
Jul 17 14:37:14.436660 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.436643    2561 scope.go:117] "RemoveContainer" containerID="1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830"
Jul 17 14:37:14.444992 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.444969    2561 scope.go:117] "RemoveContainer" containerID="4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7"
Jul 17 14:37:14.445571 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:37:14.445548    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7\": container with ID starting with 4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7 not found: ID does not exist" containerID="4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7"
Jul 17 14:37:14.445671 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.445579    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7"} err="failed to get container status \"4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7\": rpc error: code = NotFound desc = could not find container \"4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7\": container with ID starting with 4a7f71dfae4efa0ac6512810cb8ca44232f624bdde209bdb309488558b1b91a7 not found: ID does not exist"
Jul 17 14:37:14.445671 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.445596    2561 scope.go:117] "RemoveContainer" containerID="1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830"
Jul 17 14:37:14.445884 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:37:14.445865    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830\": container with ID starting with 1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830 not found: ID does not exist" containerID="1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830"
Jul 17 14:37:14.445930 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.445891    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830"} err="failed to get container status \"1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830\": rpc error: code = NotFound desc = could not find container \"1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830\": container with ID starting with 1c0ecea056c3ca099b504c50fdfccb3d83f8306a4dc56bc493469eb15739e830 not found: ID does not exist"
Jul 17 14:37:14.454238 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.454209    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"]
Jul 17 14:37:14.458189 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.458165    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8b734e-openshift-default-d686cccbf-tqs6j"]
Jul 17 14:37:14.772482 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:14.772401    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" path="/var/lib/kubelet/pods/6de929b3-0d80-450f-9875-0fb8ee8fff63/volumes"
Jul 17 14:37:15.243676 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:15.243646    2561 scope.go:117] "RemoveContainer" containerID="83068dfcf5875c49180391cd0f8c3b8600ebdecc70b540b22ba42181a13cb027"
Jul 17 14:37:37.224818 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:37.224716    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"]
Jul 17 14:37:37.225337 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:37.225088    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy" containerID="cri-o://357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646" gracePeriod=30
Jul 17 14:37:38.080243 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.080208    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"]
Jul 17 14:37:38.080837 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.080821    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerName="revoke-keys"
Jul 17 14:37:38.080884 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.080841    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerName="revoke-keys"
Jul 17 14:37:38.080884 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.080859    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerName="revoke-keys"
Jul 17 14:37:38.080884 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.080869    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerName="revoke-keys"
Jul 17 14:37:38.080884 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.080879    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy"
Jul 17 14:37:38.081041 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.080888    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy"
Jul 17 14:37:38.081041 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.080913    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerName="revoke-keys"
Jul 17 14:37:38.081041 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.080920    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerName="revoke-keys"
Jul 17 14:37:38.081041 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.081039    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerName="revoke-keys"
Jul 17 14:37:38.081174 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.081049    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerName="revoke-keys"
Jul 17 14:37:38.081174 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.081060    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy"
Jul 17 14:37:38.084841 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.084822    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.088189 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.088174    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-244a73-openshift-default-dockercfg-4dpwh\""
Jul 17 14:37:38.099842 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.099820    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk6bb\" (UniqueName: \"kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-kube-api-access-kk6bb\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.099945 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.099853    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-podinfo\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.099945 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.099872    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-envoy\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.099945 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.099889    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-data\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.100078 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.099966    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istiod-ca-cert\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.100078 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.100032    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-token\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.100078 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.100075    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-credential-socket\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.100177 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.100130    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-socket\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.100177 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.100167    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-certs\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.114888 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.114854    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"]
Jul 17 14:37:38.201328 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201292    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-socket\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.201518 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201346    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-certs\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.201518 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201398    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6bb\" (UniqueName: \"kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-kube-api-access-kk6bb\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.201518 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201421    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-podinfo\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.201692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201592    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-envoy\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.201692 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201641    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-data\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.201804 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201738    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-socket\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.201804 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201757    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istiod-ca-cert\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.201891 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201807    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-token\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.201891 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201872    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-credential-socket\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.201983 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201804    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-certs\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.202094 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.201994    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-data\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.202396 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.202304    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-credential-socket\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.202568 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.202545    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istiod-ca-cert\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.203986 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.203962    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-envoy\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.204677 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.204659    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-podinfo\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.212433 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.212399    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk6bb\" (UniqueName: \"kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-kube-api-access-kk6bb\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.212831 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.212810    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-token\") pod \"e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") " pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.397886 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.397784    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:38.566020 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.565974    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"]
Jul 17 14:37:38.568862 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:37:38.568831    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d74494_e49e_4741_9bbb_ed7cdd22b23f.slice/crio-bb04c5631bfc79139f42fc11ea98a9e2d25271b388d17ab3b2e7157b95670d3c WatchSource:0}: Error finding container bb04c5631bfc79139f42fc11ea98a9e2d25271b388d17ab3b2e7157b95670d3c: Status 404 returned error can't find the container with id bb04c5631bfc79139f42fc11ea98a9e2d25271b388d17ab3b2e7157b95670d3c
Jul 17 14:37:38.571231 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.571194    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:37:38.571330 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.571282    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:37:38.571330 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:38.571322    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:37:39.567324 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:39.567292    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" event={"ID":"99d74494-e49e-4741-9bbb-ed7cdd22b23f","Type":"ContainerStarted","Data":"e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215"}
Jul 17 14:37:39.567694 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:39.567342    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" event={"ID":"99d74494-e49e-4741-9bbb-ed7cdd22b23f","Type":"ContainerStarted","Data":"bb04c5631bfc79139f42fc11ea98a9e2d25271b388d17ab3b2e7157b95670d3c"}
Jul 17 14:37:39.621116 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:39.621060    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" podStartSLOduration=1.621040932 podStartE2EDuration="1.621040932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:37:39.620033489 +0000 UTC m=+3145.326480319" watchObservedRunningTime="2026-07-17 14:37:39.621040932 +0000 UTC m=+3145.327487759"
Jul 17 14:37:40.398511 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:40.398474    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:40.403824 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:40.403800    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:40.572793 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:40.572760    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:40.573643 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:40.573626    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:42.502722 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.502699    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:37:42.545766 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.545729    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istiod-ca-cert\") pod \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") "
Jul 17 14:37:42.545935 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.545781    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-podinfo\") pod \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") "
Jul 17 14:37:42.545935 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.545829    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-certs\") pod \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") "
Jul 17 14:37:42.545935 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.545878    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-token\") pod \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") "
Jul 17 14:37:42.545935 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.545905    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-data\") pod \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") "
Jul 17 14:37:42.546220 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.545933    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnqvt\" (UniqueName: \"kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-kube-api-access-tnqvt\") pod \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") "
Jul 17 14:37:42.546220 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546045    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-socket\") pod \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") "
Jul 17 14:37:42.546220 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546090    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-envoy\") pod \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") "
Jul 17 14:37:42.546220 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546150    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-credential-socket\") pod \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\" (UID: \"ecd82ab1-9dc9-4fe9-a76e-b9773df20106\") "
Jul 17 14:37:42.546220 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546167    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ecd82ab1-9dc9-4fe9-a76e-b9773df20106" (UID: "ecd82ab1-9dc9-4fe9-a76e-b9773df20106"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:37:42.546481 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546380    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ecd82ab1-9dc9-4fe9-a76e-b9773df20106" (UID: "ecd82ab1-9dc9-4fe9-a76e-b9773df20106"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:37:42.546481 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546193    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ecd82ab1-9dc9-4fe9-a76e-b9773df20106" (UID: "ecd82ab1-9dc9-4fe9-a76e-b9773df20106"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:37:42.546481 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546467    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ecd82ab1-9dc9-4fe9-a76e-b9773df20106" (UID: "ecd82ab1-9dc9-4fe9-a76e-b9773df20106"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:37:42.546638 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546580    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:42.546638 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546598    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:42.546638 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546613    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:42.546638 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.546628    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:42.547094 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.547054    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-data" (OuterVolumeSpecName: "istio-data") pod "ecd82ab1-9dc9-4fe9-a76e-b9773df20106" (UID: "ecd82ab1-9dc9-4fe9-a76e-b9773df20106"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:37:42.548783 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.548638    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ecd82ab1-9dc9-4fe9-a76e-b9773df20106" (UID: "ecd82ab1-9dc9-4fe9-a76e-b9773df20106"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:37:42.548952 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.548820    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ecd82ab1-9dc9-4fe9-a76e-b9773df20106" (UID: "ecd82ab1-9dc9-4fe9-a76e-b9773df20106"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:37:42.549598 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.549556    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-token" (OuterVolumeSpecName: "istio-token") pod "ecd82ab1-9dc9-4fe9-a76e-b9773df20106" (UID: "ecd82ab1-9dc9-4fe9-a76e-b9773df20106"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:37:42.549788 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.549767    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-kube-api-access-tnqvt" (OuterVolumeSpecName: "kube-api-access-tnqvt") pod "ecd82ab1-9dc9-4fe9-a76e-b9773df20106" (UID: "ecd82ab1-9dc9-4fe9-a76e-b9773df20106"). InnerVolumeSpecName "kube-api-access-tnqvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:37:42.586502 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.586461    2561 generic.go:356] "Generic (PLEG): container finished" podID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerID="357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646" exitCode=0
Jul 17 14:37:42.586646 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.586512    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"
Jul 17 14:37:42.586646 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.586552    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" event={"ID":"ecd82ab1-9dc9-4fe9-a76e-b9773df20106","Type":"ContainerDied","Data":"357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646"}
Jul 17 14:37:42.586646 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.586614    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr" event={"ID":"ecd82ab1-9dc9-4fe9-a76e-b9773df20106","Type":"ContainerDied","Data":"c294bd9b8d1af1374cdb3fca47fe8367d1d9063b567ce7bc7dd882d925f09e85"}
Jul 17 14:37:42.586646 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.586643    2561 scope.go:117] "RemoveContainer" containerID="357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646"
Jul 17 14:37:42.596258 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.596237    2561 scope.go:117] "RemoveContainer" containerID="9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567"
Jul 17 14:37:42.604277 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.604256    2561 scope.go:117] "RemoveContainer" containerID="357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646"
Jul 17 14:37:42.604511 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:37:42.604494    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646\": container with ID starting with 357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646 not found: ID does not exist" containerID="357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646"
Jul 17 14:37:42.604556 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.604518    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646"} err="failed to get container status \"357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646\": rpc error: code = NotFound desc = could not find container \"357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646\": container with ID starting with 357f026adbd589def0fbce8528de3d83ac7bb2ad84a85ae05939597090e6d646 not found: ID does not exist"
Jul 17 14:37:42.604556 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.604536    2561 scope.go:117] "RemoveContainer" containerID="9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567"
Jul 17 14:37:42.604778 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:37:42.604760    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567\": container with ID starting with 9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567 not found: ID does not exist" containerID="9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567"
Jul 17 14:37:42.604842 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.604783    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567"} err="failed to get container status \"9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567\": rpc error: code = NotFound desc = could not find container \"9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567\": container with ID starting with 9e9e820ed30179eb57b975f16cdc1002702d43623b43edfc66a71a288359d567 not found: ID does not exist"
Jul 17 14:37:42.620020 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.619978    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"]
Jul 17 14:37:42.626787 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.626758    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b55149-openshift-default-7fcb9d7c49-6dbqr"]
Jul 17 14:37:42.648222 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.648190    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:42.648804 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.648778    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:42.648935 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.648809    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnqvt\" (UniqueName: \"kubernetes.io/projected/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-kube-api-access-tnqvt\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:42.648935 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.648827    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:42.648935 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.648844    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecd82ab1-9dc9-4fe9-a76e-b9773df20106-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:37:42.773428 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:42.773328    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" path="/var/lib/kubelet/pods/ecd82ab1-9dc9-4fe9-a76e-b9773df20106/volumes"
Jul 17 14:37:46.916201 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916172    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"]
Jul 17 14:37:46.916763 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916743    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy"
Jul 17 14:37:46.916868 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916766    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy"
Jul 17 14:37:46.916868 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916787    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy"
Jul 17 14:37:46.916868 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916795    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy"
Jul 17 14:37:46.916868 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916829    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy"
Jul 17 14:37:46.916868 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916837    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy"
Jul 17 14:37:46.917154 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916940    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy"
Jul 17 14:37:46.917154 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916956    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecd82ab1-9dc9-4fe9-a76e-b9773df20106" containerName="istio-proxy"
Jul 17 14:37:46.917154 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916968    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9e06a34-b66f-4380-b562-30a6c18a5614" containerName="revoke-keys"
Jul 17 14:37:46.917154 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.916980    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6de929b3-0d80-450f-9875-0fb8ee8fff63" containerName="istio-proxy"
Jul 17 14:37:46.921793 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.921759    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:46.925385 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.925360    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-9eb655-openshift-default-dockercfg-rr5rv\""
Jul 17 14:37:46.942479 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.942453    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"]
Jul 17 14:37:46.993630 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.993596    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-envoy\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:46.993989 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.993958    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e37031-2b99-43ea-8069-f4623447bf26-istio-podinfo\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:46.994136 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.993998    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-certs\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:46.994136 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.994068    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-istio-token\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:46.994136 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.994092    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-credential-socket\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:46.994136 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.994126    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-socket\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:46.994313 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.994144    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcddt\" (UniqueName: \"kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-kube-api-access-vcddt\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:46.994313 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.994185    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-data\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:46.994313 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:46.994261    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e37031-2b99-43ea-8069-f4623447bf26-istiod-ca-cert\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.094938 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.094900    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e37031-2b99-43ea-8069-f4623447bf26-istiod-ca-cert\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095163 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.094965    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-envoy\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095163 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095066    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e37031-2b99-43ea-8069-f4623447bf26-istio-podinfo\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095163 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095095    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-certs\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095163 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095118    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-istio-token\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095163 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095145    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-credential-socket\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095444 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095182    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-socket\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095444 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095208    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcddt\" (UniqueName: \"kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-kube-api-access-vcddt\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095444 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095258    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-data\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095619 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095542    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-certs\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095701 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095668    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e37031-2b99-43ea-8069-f4623447bf26-istiod-ca-cert\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095769 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-socket\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095769 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095755    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-credential-socket\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.095886 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.095840    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-data\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.097507 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.097487    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-envoy\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.097778 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.097761    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e37031-2b99-43ea-8069-f4623447bf26-istio-podinfo\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.115032 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.114978    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcddt\" (UniqueName: \"kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-kube-api-access-vcddt\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.116667 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.116645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-istio-token\") pod \"e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") " pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.234560 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.234532    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:47.383648 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.383621    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"]
Jul 17 14:37:47.384582 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:37:47.384556    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e37031_2b99_43ea_8069_f4623447bf26.slice/crio-8b0ccefb5155afc27bdd0ebd12de05d56c5e4b3f9e279b5372c79b5b237ef74f WatchSource:0}: Error finding container 8b0ccefb5155afc27bdd0ebd12de05d56c5e4b3f9e279b5372c79b5b237ef74f: Status 404 returned error can't find the container with id 8b0ccefb5155afc27bdd0ebd12de05d56c5e4b3f9e279b5372c79b5b237ef74f
Jul 17 14:37:47.386744 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.386710    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:37:47.386815 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.386779    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:37:47.386815 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.386811    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:37:47.625950 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.625833    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" event={"ID":"f0e37031-2b99-43ea-8069-f4623447bf26","Type":"ContainerStarted","Data":"ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e"}
Jul 17 14:37:47.625950 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.625901    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" event={"ID":"f0e37031-2b99-43ea-8069-f4623447bf26","Type":"ContainerStarted","Data":"8b0ccefb5155afc27bdd0ebd12de05d56c5e4b3f9e279b5372c79b5b237ef74f"}
Jul 17 14:37:47.652379 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:47.652331    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" podStartSLOduration=1.652315005 podStartE2EDuration="1.652315005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:37:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:37:47.649697147 +0000 UTC m=+3153.356144007" watchObservedRunningTime="2026-07-17 14:37:47.652315005 +0000 UTC m=+3153.358761839"
Jul 17 14:37:48.235529 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:48.235494    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:48.240157 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:48.240134    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:48.636325 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:48.636227    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:48.638401 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:48.638372    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:37:55.574380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:55.574262    2561 patch_prober.go:28] interesting pod/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Jul 17 14:37:55.574380 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:55.574323    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Jul 17 14:37:55.680193 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:55.680162    2561 generic.go:356] "Generic (PLEG): container finished" podID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerID="e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215" exitCode=0
Jul 17 14:37:55.680361 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:55.680197    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" event={"ID":"99d74494-e49e-4741-9bbb-ed7cdd22b23f","Type":"ContainerDied","Data":"e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215"}
Jul 17 14:37:55.680606 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:55.680593    2561 scope.go:117] "RemoveContainer" containerID="e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215"
Jul 17 14:37:55.681675 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:55.681636    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:37:55.681745 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:55.681718    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:37:55.681782 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:55.681754    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:37:56.687786 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:56.687750    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" event={"ID":"99d74494-e49e-4741-9bbb-ed7cdd22b23f","Type":"ContainerStarted","Data":"4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1"}
Jul 17 14:37:57.398158 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:57.398129    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:37:57.398529 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:57.398507    2561 patch_prober.go:28] interesting pod/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Jul 17 14:37:57.398578 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:57.398558    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Jul 17 14:37:58.398218 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:58.398184    2561 patch_prober.go:28] interesting pod/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Jul 17 14:37:58.398610 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:58.398235    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Jul 17 14:37:59.398280 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:59.398237    2561 patch_prober.go:28] interesting pod/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Jul 17 14:37:59.398679 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:37:59.398310    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Jul 17 14:38:00.399131 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:00.399091    2561 patch_prober.go:28] interesting pod/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Jul 17 14:38:00.399542 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:00.399147    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Jul 17 14:38:01.403719 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:01.403681    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:38:01.404229 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:01.404209    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:38:01.404926 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:01.404907    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:38:03.637086 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:03.637052    2561 patch_prober.go:28] interesting pod/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Jul 17 14:38:03.637495 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:03.637107    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Jul 17 14:38:03.728925 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:03.728888    2561 generic.go:356] "Generic (PLEG): container finished" podID="f0e37031-2b99-43ea-8069-f4623447bf26" containerID="ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e" exitCode=0
Jul 17 14:38:03.729390 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:03.728967    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" event={"ID":"f0e37031-2b99-43ea-8069-f4623447bf26","Type":"ContainerDied","Data":"ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e"}
Jul 17 14:38:03.729543 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:03.729524    2561 scope.go:117] "RemoveContainer" containerID="ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e"
Jul 17 14:38:03.730809 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:03.730770    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:38:03.730897 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:03.730875    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:38:03.730934 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:03.730921    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 14:38:04.740830 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:04.739797    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" event={"ID":"f0e37031-2b99-43ea-8069-f4623447bf26","Type":"ContainerStarted","Data":"502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a"}
Jul 17 14:38:05.235171 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:05.235136    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:38:05.235571 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:05.235476    2561 patch_prober.go:28] interesting pod/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Jul 17 14:38:05.235735 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:05.235711    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Jul 17 14:38:06.235931 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:06.235887    2561 patch_prober.go:28] interesting pod/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Jul 17 14:38:06.236464 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:06.235966    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Jul 17 14:38:07.235785 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:07.235748    2561 patch_prober.go:28] interesting pod/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Jul 17 14:38:07.236116 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:07.236084    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Jul 17 14:38:08.235980 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:08.235931    2561 patch_prober.go:28] interesting pod/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Jul 17 14:38:08.236218 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:08.236036    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Jul 17 14:38:09.239867 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:09.239839    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:38:09.240262 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:09.240129    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:38:09.240910 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:38:09.240889    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:40:56.286140 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.286036    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"]
Jul 17 14:40:56.290085 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.290066    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.292745 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.292725    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"kube-root-ca.crt\""
Jul 17 14:40:56.293754 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.293729    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"sub-test-model-244a73-kserve-self-signed-certs\""
Jul 17 14:40:56.293864 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.293761    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"default-dockercfg-hl8w5\""
Jul 17 14:40:56.293864 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.293763    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-244a73\"/\"openshift-service-ca.crt\""
Jul 17 14:40:56.303555 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.303529    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"]
Jul 17 14:40:56.327390 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.327358    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-dshm\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.327523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.327406    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-home\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.327523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.327442    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-model-cache\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.327523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.327462    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f5371a8-10df-4c36-9c1a-583c312ff548-tls-certs\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.327523 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.327486    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-tmp-dir\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.327753 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.327619    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6dcp\" (UniqueName: \"kubernetes.io/projected/5f5371a8-10df-4c36-9c1a-583c312ff548-kube-api-access-b6dcp\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.428449 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.428409    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f5371a8-10df-4c36-9c1a-583c312ff548-tls-certs\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.428449 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.428451    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-tmp-dir\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.428689 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.428517    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6dcp\" (UniqueName: \"kubernetes.io/projected/5f5371a8-10df-4c36-9c1a-583c312ff548-kube-api-access-b6dcp\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.428689 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.428575    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-dshm\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.428689 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.428591    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-home\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.428689 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.428610    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-model-cache\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.428991 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.428965    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-tmp-dir\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.429085 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.429032    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-model-cache\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.429145 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.429104    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-home\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.430900 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.430876    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-dshm\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.431435 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.431412    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f5371a8-10df-4c36-9c1a-583c312ff548-tls-certs\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.439488 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.439463    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6dcp\" (UniqueName: \"kubernetes.io/projected/5f5371a8-10df-4c36-9c1a-583c312ff548-kube-api-access-b6dcp\") pod \"sub-test-model-244a73-kserve-6b564ffd57-gj64s\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") " pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.602285 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.602205    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:40:56.748034 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:56.747863    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"]
Jul 17 14:40:56.752031 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:40:56.751900    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f5371a8_10df_4c36_9c1a_583c312ff548.slice/crio-f3d9bb341dc317c849b6884acba32a9c1070888ae893c3b5fb05321472e0b57b WatchSource:0}: Error finding container f3d9bb341dc317c849b6884acba32a9c1070888ae893c3b5fb05321472e0b57b: Status 404 returned error can't find the container with id f3d9bb341dc317c849b6884acba32a9c1070888ae893c3b5fb05321472e0b57b
Jul 17 14:40:57.732000 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:57.731962    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s" event={"ID":"5f5371a8-10df-4c36-9c1a-583c312ff548","Type":"ContainerStarted","Data":"87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1"}
Jul 17 14:40:57.732496 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:57.732022    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s" event={"ID":"5f5371a8-10df-4c36-9c1a-583c312ff548","Type":"ContainerStarted","Data":"f3d9bb341dc317c849b6884acba32a9c1070888ae893c3b5fb05321472e0b57b"}
Jul 17 14:40:57.754910 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:40:57.754861    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s" podStartSLOduration=1.754845725 podStartE2EDuration="1.754845725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:40:57.752707069 +0000 UTC m=+3343.459153916" watchObservedRunningTime="2026-07-17 14:40:57.754845725 +0000 UTC m=+3343.461292580"
Jul 17 14:41:06.603291 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:41:06.603258    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:41:06.603672 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:41:06.603307    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:41:06.615712 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:41:06.615683    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:41:06.792641 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:41:06.792608    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:41:52.667884 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:41:52.667853    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"]
Jul 17 14:42:07.689418 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:07.689330    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s" podUID="5f5371a8-10df-4c36-9c1a-583c312ff548" containerName="main" containerID="cri-o://87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1" gracePeriod=45
Jul 17 14:42:07.965829 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:07.965802    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:42:08.106861 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.106831    2561 generic.go:356] "Generic (PLEG): container finished" podID="5f5371a8-10df-4c36-9c1a-583c312ff548" containerID="87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1" exitCode=0
Jul 17 14:42:08.107042 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.106877    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"
Jul 17 14:42:08.107042 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.106888    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s" event={"ID":"5f5371a8-10df-4c36-9c1a-583c312ff548","Type":"ContainerDied","Data":"87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1"}
Jul 17 14:42:08.107042 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.106919    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s" event={"ID":"5f5371a8-10df-4c36-9c1a-583c312ff548","Type":"ContainerDied","Data":"f3d9bb341dc317c849b6884acba32a9c1070888ae893c3b5fb05321472e0b57b"}
Jul 17 14:42:08.107042 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.106935    2561 scope.go:117] "RemoveContainer" containerID="87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1"
Jul 17 14:42:08.115973 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.115956    2561 scope.go:117] "RemoveContainer" containerID="87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1"
Jul 17 14:42:08.116226 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:42:08.116210    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1\": container with ID starting with 87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1 not found: ID does not exist" containerID="87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1"
Jul 17 14:42:08.116288 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.116235    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1"} err="failed to get container status \"87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1\": rpc error: code = NotFound desc = could not find container \"87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1\": container with ID starting with 87a8b3a70e7071760142b7692f9d93cd2f1489501a31f4cc6aee1b4da10b0dc1 not found: ID does not exist"
Jul 17 14:42:08.118639 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.118622    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-tmp-dir\") pod \"5f5371a8-10df-4c36-9c1a-583c312ff548\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") "
Jul 17 14:42:08.118697 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.118653    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-model-cache\") pod \"5f5371a8-10df-4c36-9c1a-583c312ff548\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") "
Jul 17 14:42:08.118697 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.118671    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f5371a8-10df-4c36-9c1a-583c312ff548-tls-certs\") pod \"5f5371a8-10df-4c36-9c1a-583c312ff548\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") "
Jul 17 14:42:08.118772 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.118695    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-home\") pod \"5f5371a8-10df-4c36-9c1a-583c312ff548\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") "
Jul 17 14:42:08.118772 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.118727    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6dcp\" (UniqueName: \"kubernetes.io/projected/5f5371a8-10df-4c36-9c1a-583c312ff548-kube-api-access-b6dcp\") pod \"5f5371a8-10df-4c36-9c1a-583c312ff548\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") "
Jul 17 14:42:08.118772 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.118765    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-dshm\") pod \"5f5371a8-10df-4c36-9c1a-583c312ff548\" (UID: \"5f5371a8-10df-4c36-9c1a-583c312ff548\") "
Jul 17 14:42:08.118938 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.118909    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-model-cache" (OuterVolumeSpecName: "model-cache") pod "5f5371a8-10df-4c36-9c1a-583c312ff548" (UID: "5f5371a8-10df-4c36-9c1a-583c312ff548"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:42:08.119021 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.118977    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5f5371a8-10df-4c36-9c1a-583c312ff548" (UID: "5f5371a8-10df-4c36-9c1a-583c312ff548"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:42:08.119021 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.118993    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-home" (OuterVolumeSpecName: "home") pod "5f5371a8-10df-4c36-9c1a-583c312ff548" (UID: "5f5371a8-10df-4c36-9c1a-583c312ff548"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:42:08.119200 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.119178    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-tmp-dir\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:42:08.119200 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.119198    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-model-cache\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:42:08.119539 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.119207    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-home\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:42:08.120997 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.120970    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-dshm" (OuterVolumeSpecName: "dshm") pod "5f5371a8-10df-4c36-9c1a-583c312ff548" (UID: "5f5371a8-10df-4c36-9c1a-583c312ff548"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:42:08.121134 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.121050    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f5371a8-10df-4c36-9c1a-583c312ff548-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5f5371a8-10df-4c36-9c1a-583c312ff548" (UID: "5f5371a8-10df-4c36-9c1a-583c312ff548"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 14:42:08.121134 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.121112    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f5371a8-10df-4c36-9c1a-583c312ff548-kube-api-access-b6dcp" (OuterVolumeSpecName: "kube-api-access-b6dcp") pod "5f5371a8-10df-4c36-9c1a-583c312ff548" (UID: "5f5371a8-10df-4c36-9c1a-583c312ff548"). InnerVolumeSpecName "kube-api-access-b6dcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:42:08.220721 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.220664    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f5371a8-10df-4c36-9c1a-583c312ff548-tls-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:42:08.220721 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.220692    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6dcp\" (UniqueName: \"kubernetes.io/projected/5f5371a8-10df-4c36-9c1a-583c312ff548-kube-api-access-b6dcp\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:42:08.220721 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.220702    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f5371a8-10df-4c36-9c1a-583c312ff548-dshm\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:42:08.431233 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.431197    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"]
Jul 17 14:42:08.433362 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.433335    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-244a73/sub-test-model-244a73-kserve-6b564ffd57-gj64s"]
Jul 17 14:42:08.772795 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:42:08.772765    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f5371a8-10df-4c36-9c1a-583c312ff548" path="/var/lib/kubelet/pods/5f5371a8-10df-4c36-9c1a-583c312ff548/volumes"
Jul 17 14:43:47.847048 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:47.846945    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7683-140031006684976"]
Jul 17 14:43:47.847526 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:47.847508    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f5371a8-10df-4c36-9c1a-583c312ff548" containerName="main"
Jul 17 14:43:47.847576 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:47.847528    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f5371a8-10df-4c36-9c1a-583c312ff548" containerName="main"
Jul 17 14:43:47.847626 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:47.847605    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f5371a8-10df-4c36-9c1a-583c312ff548" containerName="main"
Jul 17 14:43:47.850636 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:47.850620    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031006684976"
Jul 17 14:43:47.853679 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:47.853660    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4cxcv\""
Jul 17 14:43:47.863440 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:47.863420    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031006684976"]
Jul 17 14:43:47.980801 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:47.980759    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dz9f\" (UniqueName: \"kubernetes.io/projected/428cba90-4989-4f23-8ba0-712404331a03-kube-api-access-2dz9f\") pod \"test-curl-7683-140031006684976\" (UID: \"428cba90-4989-4f23-8ba0-712404331a03\") " pod="opendatahub/test-curl-7683-140031006684976"
Jul 17 14:43:48.082237 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:48.082206    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dz9f\" (UniqueName: \"kubernetes.io/projected/428cba90-4989-4f23-8ba0-712404331a03-kube-api-access-2dz9f\") pod \"test-curl-7683-140031006684976\" (UID: \"428cba90-4989-4f23-8ba0-712404331a03\") " pod="opendatahub/test-curl-7683-140031006684976"
Jul 17 14:43:48.092956 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:48.092935    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dz9f\" (UniqueName: \"kubernetes.io/projected/428cba90-4989-4f23-8ba0-712404331a03-kube-api-access-2dz9f\") pod \"test-curl-7683-140031006684976\" (UID: \"428cba90-4989-4f23-8ba0-712404331a03\") " pod="opendatahub/test-curl-7683-140031006684976"
Jul 17 14:43:48.159840 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:48.159780    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031006684976"
Jul 17 14:43:48.314854 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:48.314825    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031006684976"]
Jul 17 14:43:48.315261 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:43:48.315223    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod428cba90_4989_4f23_8ba0_712404331a03.slice/crio-65d48c6bfeaef59a64c5112a0b2f810d70214230185ed8429b1d70b9eca1e3b6 WatchSource:0}: Error finding container 65d48c6bfeaef59a64c5112a0b2f810d70214230185ed8429b1d70b9eca1e3b6: Status 404 returned error can't find the container with id 65d48c6bfeaef59a64c5112a0b2f810d70214230185ed8429b1d70b9eca1e3b6
Jul 17 14:43:48.316672 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:48.316656    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 14:43:48.653158 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:48.653124    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031006684976" event={"ID":"428cba90-4989-4f23-8ba0-712404331a03","Type":"ContainerStarted","Data":"65d48c6bfeaef59a64c5112a0b2f810d70214230185ed8429b1d70b9eca1e3b6"}
Jul 17 14:43:49.659619 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:49.659538    2561 generic.go:356] "Generic (PLEG): container finished" podID="428cba90-4989-4f23-8ba0-712404331a03" containerID="54e8431db1ca4f785fd8a598623d6ef840eb21283cda925677846f526fff5c40" exitCode=0
Jul 17 14:43:49.659984 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:49.659619    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031006684976" event={"ID":"428cba90-4989-4f23-8ba0-712404331a03","Type":"ContainerDied","Data":"54e8431db1ca4f785fd8a598623d6ef840eb21283cda925677846f526fff5c40"}
Jul 17 14:43:50.805634 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:50.805611    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031006684976"
Jul 17 14:43:50.853070 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:50.853044    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7683-140031006684976_428cba90-4989-4f23-8ba0-712404331a03/test-curl-7683-140031006684976/0.log"
Jul 17 14:43:50.879426 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:50.879401    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7683-140031006684976"]
Jul 17 14:43:50.883832 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:50.883813    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7683-140031006684976"]
Jul 17 14:43:50.910939 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:50.910919    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dz9f\" (UniqueName: \"kubernetes.io/projected/428cba90-4989-4f23-8ba0-712404331a03-kube-api-access-2dz9f\") pod \"428cba90-4989-4f23-8ba0-712404331a03\" (UID: \"428cba90-4989-4f23-8ba0-712404331a03\") "
Jul 17 14:43:50.913217 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:50.913189    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/428cba90-4989-4f23-8ba0-712404331a03-kube-api-access-2dz9f" (OuterVolumeSpecName: "kube-api-access-2dz9f") pod "428cba90-4989-4f23-8ba0-712404331a03" (UID: "428cba90-4989-4f23-8ba0-712404331a03"). InnerVolumeSpecName "kube-api-access-2dz9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:43:51.012958 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.012901    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dz9f\" (UniqueName: \"kubernetes.io/projected/428cba90-4989-4f23-8ba0-712404331a03-kube-api-access-2dz9f\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:43:51.023728 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.023697    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7683-140031008944560"]
Jul 17 14:43:51.024419 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.024400    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="428cba90-4989-4f23-8ba0-712404331a03" containerName="test-curl-7683-140031006684976"
Jul 17 14:43:51.024476 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.024424    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="428cba90-4989-4f23-8ba0-712404331a03" containerName="test-curl-7683-140031006684976"
Jul 17 14:43:51.024596 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.024577    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="428cba90-4989-4f23-8ba0-712404331a03" containerName="test-curl-7683-140031006684976"
Jul 17 14:43:51.027779 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.027764    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008944560"
Jul 17 14:43:51.036853 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.036832    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031008944560"]
Jul 17 14:43:51.113759 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.113731    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m8jb\" (UniqueName: \"kubernetes.io/projected/752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a-kube-api-access-4m8jb\") pod \"test-curl-7683-140031008944560\" (UID: \"752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a\") " pod="opendatahub/test-curl-7683-140031008944560"
Jul 17 14:43:51.215069 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.214973    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m8jb\" (UniqueName: \"kubernetes.io/projected/752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a-kube-api-access-4m8jb\") pod \"test-curl-7683-140031008944560\" (UID: \"752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a\") " pod="opendatahub/test-curl-7683-140031008944560"
Jul 17 14:43:51.227782 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.227752    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m8jb\" (UniqueName: \"kubernetes.io/projected/752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a-kube-api-access-4m8jb\") pod \"test-curl-7683-140031008944560\" (UID: \"752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a\") " pod="opendatahub/test-curl-7683-140031008944560"
Jul 17 14:43:51.337395 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.337367    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008944560"
Jul 17 14:43:51.492496 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.492470    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031008944560"]
Jul 17 14:43:51.494624 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:43:51.494600    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod752a75bf_a30c_4af1_9ee5_d4eb8cc9eb5a.slice/crio-efabc7439c80e1308eac388790bb3278a30e4dc9dbe59c0fe091262bd0e17aa8 WatchSource:0}: Error finding container efabc7439c80e1308eac388790bb3278a30e4dc9dbe59c0fe091262bd0e17aa8: Status 404 returned error can't find the container with id efabc7439c80e1308eac388790bb3278a30e4dc9dbe59c0fe091262bd0e17aa8
Jul 17 14:43:51.671847 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.671814    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031008944560" event={"ID":"752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a","Type":"ContainerStarted","Data":"efabc7439c80e1308eac388790bb3278a30e4dc9dbe59c0fe091262bd0e17aa8"}
Jul 17 14:43:51.672906 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.672887    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031006684976"
Jul 17 14:43:51.673025 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:51.672936    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65d48c6bfeaef59a64c5112a0b2f810d70214230185ed8429b1d70b9eca1e3b6"
Jul 17 14:43:51.683486 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:43:51.683460    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7683-140031006684976\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="428cba90-4989-4f23-8ba0-712404331a03" pod="opendatahub/test-curl-7683-140031006684976"
Jul 17 14:43:52.680271 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:52.680240    2561 generic.go:356] "Generic (PLEG): container finished" podID="752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a" containerID="c3b075e1ae130340102304a776c874141015a7e2e972bca8306325e6d4aff112" exitCode=0
Jul 17 14:43:52.680636 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:52.680340    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031008944560" event={"ID":"752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a","Type":"ContainerDied","Data":"c3b075e1ae130340102304a776c874141015a7e2e972bca8306325e6d4aff112"}
Jul 17 14:43:52.700481 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:43:52.700449    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7683-140031006684976\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="428cba90-4989-4f23-8ba0-712404331a03" pod="opendatahub/test-curl-7683-140031006684976"
Jul 17 14:43:52.772250 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:52.772220    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="428cba90-4989-4f23-8ba0-712404331a03" path="/var/lib/kubelet/pods/428cba90-4989-4f23-8ba0-712404331a03/volumes"
Jul 17 14:43:53.828738 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:53.828713    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008944560"
Jul 17 14:43:53.852606 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:53.852584    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7683-140031008944560_752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a/test-curl-7683-140031008944560/0.log"
Jul 17 14:43:53.887417 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:53.887392    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7683-140031008944560"]
Jul 17 14:43:53.892830 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:53.892807    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7683-140031008944560"]
Jul 17 14:43:53.940209 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:53.940139    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4m8jb\" (UniqueName: \"kubernetes.io/projected/752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a-kube-api-access-4m8jb\") pod \"752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a\" (UID: \"752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a\") "
Jul 17 14:43:53.942313 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:53.942285    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a-kube-api-access-4m8jb" (OuterVolumeSpecName: "kube-api-access-4m8jb") pod "752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a" (UID: "752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a"). InnerVolumeSpecName "kube-api-access-4m8jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:43:54.041438 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.041408    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m8jb\" (UniqueName: \"kubernetes.io/projected/752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a-kube-api-access-4m8jb\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:43:54.057754 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.057723    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7683-140031008928080"]
Jul 17 14:43:54.058218 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.058203    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a" containerName="test-curl-7683-140031008944560"
Jul 17 14:43:54.058271 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.058220    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a" containerName="test-curl-7683-140031008944560"
Jul 17 14:43:54.058307 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.058299    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a" containerName="test-curl-7683-140031008944560"
Jul 17 14:43:54.061399 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.061384    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:54.067872 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.067849    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031008928080"]
Jul 17 14:43:54.142664 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.142628    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkn78\" (UniqueName: \"kubernetes.io/projected/5163f5a1-186f-4588-9c1f-520b89b9aaeb-kube-api-access-qkn78\") pod \"test-curl-7683-140031008928080\" (UID: \"5163f5a1-186f-4588-9c1f-520b89b9aaeb\") " pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:54.243713 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.243638    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkn78\" (UniqueName: \"kubernetes.io/projected/5163f5a1-186f-4588-9c1f-520b89b9aaeb-kube-api-access-qkn78\") pod \"test-curl-7683-140031008928080\" (UID: \"5163f5a1-186f-4588-9c1f-520b89b9aaeb\") " pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:54.255639 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.255615    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkn78\" (UniqueName: \"kubernetes.io/projected/5163f5a1-186f-4588-9c1f-520b89b9aaeb-kube-api-access-qkn78\") pod \"test-curl-7683-140031008928080\" (UID: \"5163f5a1-186f-4588-9c1f-520b89b9aaeb\") " pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:54.372972 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.372936    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:54.512941 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.512839    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031008928080"]
Jul 17 14:43:54.515313 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:43:54.515284    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5163f5a1_186f_4588_9c1f_520b89b9aaeb.slice/crio-3539974558ae4d731b9b94c6db68d89a114dea13e1a9a16aee07aff2f4b2b7cc WatchSource:0}: Error finding container 3539974558ae4d731b9b94c6db68d89a114dea13e1a9a16aee07aff2f4b2b7cc: Status 404 returned error can't find the container with id 3539974558ae4d731b9b94c6db68d89a114dea13e1a9a16aee07aff2f4b2b7cc
Jul 17 14:43:54.692624 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.692595    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008944560"
Jul 17 14:43:54.692624 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.692618    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="efabc7439c80e1308eac388790bb3278a30e4dc9dbe59c0fe091262bd0e17aa8"
Jul 17 14:43:54.693781 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.693757    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031008928080" event={"ID":"5163f5a1-186f-4588-9c1f-520b89b9aaeb","Type":"ContainerStarted","Data":"3539974558ae4d731b9b94c6db68d89a114dea13e1a9a16aee07aff2f4b2b7cc"}
Jul 17 14:43:54.703492 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:43:54.703468    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7683-140031008944560\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a" pod="opendatahub/test-curl-7683-140031008944560"
Jul 17 14:43:54.771483 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:43:54.771412    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7683-140031008944560\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a" pod="opendatahub/test-curl-7683-140031008944560"
Jul 17 14:43:54.772484 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:54.772424    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a" path="/var/lib/kubelet/pods/752a75bf-a30c-4af1-9ee5-d4eb8cc9eb5a/volumes"
Jul 17 14:43:55.702139 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:55.702100    2561 generic.go:356] "Generic (PLEG): container finished" podID="5163f5a1-186f-4588-9c1f-520b89b9aaeb" containerID="5e287ec374de3b7b1e698e436907e46503a8ba84ef5c71e58a975bba8079dff1" exitCode=0
Jul 17 14:43:55.702542 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:55.702155    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031008928080" event={"ID":"5163f5a1-186f-4588-9c1f-520b89b9aaeb","Type":"ContainerDied","Data":"5e287ec374de3b7b1e698e436907e46503a8ba84ef5c71e58a975bba8079dff1"}
Jul 17 14:43:56.851174 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:56.851150    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:56.881383 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:56.881361    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7683-140031008928080_5163f5a1-186f-4588-9c1f-520b89b9aaeb/test-curl-7683-140031008928080/0.log"
Jul 17 14:43:56.933144 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:56.933117    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7683-140031008928080"]
Jul 17 14:43:56.937983 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:56.937962    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7683-140031008928080"]
Jul 17 14:43:56.970041 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:56.969963    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkn78\" (UniqueName: \"kubernetes.io/projected/5163f5a1-186f-4588-9c1f-520b89b9aaeb-kube-api-access-qkn78\") pod \"5163f5a1-186f-4588-9c1f-520b89b9aaeb\" (UID: \"5163f5a1-186f-4588-9c1f-520b89b9aaeb\") "
Jul 17 14:43:56.972182 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:56.972149    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5163f5a1-186f-4588-9c1f-520b89b9aaeb-kube-api-access-qkn78" (OuterVolumeSpecName: "kube-api-access-qkn78") pod "5163f5a1-186f-4588-9c1f-520b89b9aaeb" (UID: "5163f5a1-186f-4588-9c1f-520b89b9aaeb"). InnerVolumeSpecName "kube-api-access-qkn78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:43:57.070883 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.070856    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7683-140031008928080"]
Jul 17 14:43:57.071020 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.070859    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkn78\" (UniqueName: \"kubernetes.io/projected/5163f5a1-186f-4588-9c1f-520b89b9aaeb-kube-api-access-qkn78\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:43:57.071345 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.071329    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5163f5a1-186f-4588-9c1f-520b89b9aaeb" containerName="test-curl-7683-140031008928080"
Jul 17 14:43:57.071385 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.071349    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5163f5a1-186f-4588-9c1f-520b89b9aaeb" containerName="test-curl-7683-140031008928080"
Jul 17 14:43:57.071440 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.071430    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5163f5a1-186f-4588-9c1f-520b89b9aaeb" containerName="test-curl-7683-140031008928080"
Jul 17 14:43:57.074478 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.074464    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:57.084046 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.084023    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031008928080"]
Jul 17 14:43:57.171677 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.171650    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5x48\" (UniqueName: \"kubernetes.io/projected/cf450086-7ff9-49d6-a6b8-c25296c17df4-kube-api-access-w5x48\") pod \"test-curl-7683-140031008928080\" (UID: \"cf450086-7ff9-49d6-a6b8-c25296c17df4\") " pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:57.272796 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.272730    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5x48\" (UniqueName: \"kubernetes.io/projected/cf450086-7ff9-49d6-a6b8-c25296c17df4-kube-api-access-w5x48\") pod \"test-curl-7683-140031008928080\" (UID: \"cf450086-7ff9-49d6-a6b8-c25296c17df4\") " pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:57.283520 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.283496    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5x48\" (UniqueName: \"kubernetes.io/projected/cf450086-7ff9-49d6-a6b8-c25296c17df4-kube-api-access-w5x48\") pod \"test-curl-7683-140031008928080\" (UID: \"cf450086-7ff9-49d6-a6b8-c25296c17df4\") " pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:57.385061 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.385030    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:57.522353 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.522322    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031008928080"]
Jul 17 14:43:57.524658 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:43:57.524591    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf450086_7ff9_49d6_a6b8_c25296c17df4.slice/crio-9cb61f7a53a9b294bcf421d42a91924aee48d3a3c3dc5d5369c81829579cd003 WatchSource:0}: Error finding container 9cb61f7a53a9b294bcf421d42a91924aee48d3a3c3dc5d5369c81829579cd003: Status 404 returned error can't find the container with id 9cb61f7a53a9b294bcf421d42a91924aee48d3a3c3dc5d5369c81829579cd003
Jul 17 14:43:57.718430 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.718402    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:57.718584 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.718411    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3539974558ae4d731b9b94c6db68d89a114dea13e1a9a16aee07aff2f4b2b7cc"
Jul 17 14:43:57.719750 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.719723    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031008928080" event={"ID":"cf450086-7ff9-49d6-a6b8-c25296c17df4","Type":"ContainerStarted","Data":"9cb61f7a53a9b294bcf421d42a91924aee48d3a3c3dc5d5369c81829579cd003"}
Jul 17 14:43:57.731205 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:57.731172    2561 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7683-140031008928080" oldPodUID="5163f5a1-186f-4588-9c1f-520b89b9aaeb" podUID="cf450086-7ff9-49d6-a6b8-c25296c17df4"
Jul 17 14:43:58.728166 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:58.728129    2561 generic.go:356] "Generic (PLEG): container finished" podID="cf450086-7ff9-49d6-a6b8-c25296c17df4" containerID="4bb41469ed6a8ee93c017112fbf487bb727d1a35629662365f8f4f6fee947447" exitCode=0
Jul 17 14:43:58.728671 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:58.728187    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031008928080" event={"ID":"cf450086-7ff9-49d6-a6b8-c25296c17df4","Type":"ContainerDied","Data":"4bb41469ed6a8ee93c017112fbf487bb727d1a35629662365f8f4f6fee947447"}
Jul 17 14:43:58.780544 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:58.780498    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5163f5a1-186f-4588-9c1f-520b89b9aaeb" path="/var/lib/kubelet/pods/5163f5a1-186f-4588-9c1f-520b89b9aaeb/volumes"
Jul 17 14:43:59.879617 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:59.879593    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:43:59.902539 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:59.902502    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7683-140031008928080_cf450086-7ff9-49d6-a6b8-c25296c17df4/test-curl-7683-140031008928080/0.log"
Jul 17 14:43:59.929728 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:59.929701    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7683-140031008928080"]
Jul 17 14:43:59.934251 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:43:59.934227    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7683-140031008928080"]
Jul 17 14:44:00.001425 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.001363    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5x48\" (UniqueName: \"kubernetes.io/projected/cf450086-7ff9-49d6-a6b8-c25296c17df4-kube-api-access-w5x48\") pod \"cf450086-7ff9-49d6-a6b8-c25296c17df4\" (UID: \"cf450086-7ff9-49d6-a6b8-c25296c17df4\") "
Jul 17 14:44:00.003561 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.003537    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf450086-7ff9-49d6-a6b8-c25296c17df4-kube-api-access-w5x48" (OuterVolumeSpecName: "kube-api-access-w5x48") pod "cf450086-7ff9-49d6-a6b8-c25296c17df4" (UID: "cf450086-7ff9-49d6-a6b8-c25296c17df4"). InnerVolumeSpecName "kube-api-access-w5x48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:44:00.062966 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.062934    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7683-140031008929088"]
Jul 17 14:44:00.063391 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.063377    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf450086-7ff9-49d6-a6b8-c25296c17df4" containerName="test-curl-7683-140031008928080"
Jul 17 14:44:00.063442 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.063393    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf450086-7ff9-49d6-a6b8-c25296c17df4" containerName="test-curl-7683-140031008928080"
Jul 17 14:44:00.063489 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.063480    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf450086-7ff9-49d6-a6b8-c25296c17df4" containerName="test-curl-7683-140031008928080"
Jul 17 14:44:00.066727 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.066712    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008929088"
Jul 17 14:44:00.074396 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.074373    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031008929088"]
Jul 17 14:44:00.102548 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.102522    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5x48\" (UniqueName: \"kubernetes.io/projected/cf450086-7ff9-49d6-a6b8-c25296c17df4-kube-api-access-w5x48\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:00.203664 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.203629    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wncwz\" (UniqueName: \"kubernetes.io/projected/99d09cdc-b706-4a52-87e0-041809ee26bf-kube-api-access-wncwz\") pod \"test-curl-7683-140031008929088\" (UID: \"99d09cdc-b706-4a52-87e0-041809ee26bf\") " pod="opendatahub/test-curl-7683-140031008929088"
Jul 17 14:44:00.304862 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.304775    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wncwz\" (UniqueName: \"kubernetes.io/projected/99d09cdc-b706-4a52-87e0-041809ee26bf-kube-api-access-wncwz\") pod \"test-curl-7683-140031008929088\" (UID: \"99d09cdc-b706-4a52-87e0-041809ee26bf\") " pod="opendatahub/test-curl-7683-140031008929088"
Jul 17 14:44:00.314068 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.314033    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wncwz\" (UniqueName: \"kubernetes.io/projected/99d09cdc-b706-4a52-87e0-041809ee26bf-kube-api-access-wncwz\") pod \"test-curl-7683-140031008929088\" (UID: \"99d09cdc-b706-4a52-87e0-041809ee26bf\") " pod="opendatahub/test-curl-7683-140031008929088"
Jul 17 14:44:00.376483 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.376455    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008929088"
Jul 17 14:44:00.520466 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.520436    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7683-140031008929088"]
Jul 17 14:44:00.521984 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:44:00.521951    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99d09cdc_b706_4a52_87e0_041809ee26bf.slice/crio-0c00473ef42e6db68ef9ad02e32ef8431de221562771e7d50a17fe80788eeb0f WatchSource:0}: Error finding container 0c00473ef42e6db68ef9ad02e32ef8431de221562771e7d50a17fe80788eeb0f: Status 404 returned error can't find the container with id 0c00473ef42e6db68ef9ad02e32ef8431de221562771e7d50a17fe80788eeb0f
Jul 17 14:44:00.742821 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.742793    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cb61f7a53a9b294bcf421d42a91924aee48d3a3c3dc5d5369c81829579cd003"
Jul 17 14:44:00.742821 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.742793    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:44:00.744035 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.743988    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031008929088" event={"ID":"99d09cdc-b706-4a52-87e0-041809ee26bf","Type":"ContainerStarted","Data":"0c00473ef42e6db68ef9ad02e32ef8431de221562771e7d50a17fe80788eeb0f"}
Jul 17 14:44:00.753765 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:44:00.753728    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7683-140031008928080\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="cf450086-7ff9-49d6-a6b8-c25296c17df4" pod="opendatahub/test-curl-7683-140031008928080"
Jul 17 14:44:00.772943 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:00.772914    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf450086-7ff9-49d6-a6b8-c25296c17df4" path="/var/lib/kubelet/pods/cf450086-7ff9-49d6-a6b8-c25296c17df4/volumes"
Jul 17 14:44:01.750987 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:01.750950    2561 generic.go:356] "Generic (PLEG): container finished" podID="99d09cdc-b706-4a52-87e0-041809ee26bf" containerID="56f8d17e208e2af065cce25657329582e23b3e0951025129de334b7b5f175031" exitCode=0
Jul 17 14:44:01.751418 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:01.751032    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7683-140031008929088" event={"ID":"99d09cdc-b706-4a52-87e0-041809ee26bf","Type":"ContainerDied","Data":"56f8d17e208e2af065cce25657329582e23b3e0951025129de334b7b5f175031"}
Jul 17 14:44:02.902027 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:02.901978    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008929088"
Jul 17 14:44:02.936020 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:02.935973    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7683-140031008929088_99d09cdc-b706-4a52-87e0-041809ee26bf/test-curl-7683-140031008929088/0.log"
Jul 17 14:44:02.970264 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:02.970192    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7683-140031008929088"]
Jul 17 14:44:02.972654 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:02.972631    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7683-140031008929088"]
Jul 17 14:44:03.034890 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:03.034819    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wncwz\" (UniqueName: \"kubernetes.io/projected/99d09cdc-b706-4a52-87e0-041809ee26bf-kube-api-access-wncwz\") pod \"99d09cdc-b706-4a52-87e0-041809ee26bf\" (UID: \"99d09cdc-b706-4a52-87e0-041809ee26bf\") "
Jul 17 14:44:03.036879 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:03.036854    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d09cdc-b706-4a52-87e0-041809ee26bf-kube-api-access-wncwz" (OuterVolumeSpecName: "kube-api-access-wncwz") pod "99d09cdc-b706-4a52-87e0-041809ee26bf" (UID: "99d09cdc-b706-4a52-87e0-041809ee26bf"). InnerVolumeSpecName "kube-api-access-wncwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:44:03.136191 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:03.136157    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wncwz\" (UniqueName: \"kubernetes.io/projected/99d09cdc-b706-4a52-87e0-041809ee26bf-kube-api-access-wncwz\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:03.763282 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:03.763252    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7683-140031008929088"
Jul 17 14:44:03.763282 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:03.763281    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c00473ef42e6db68ef9ad02e32ef8431de221562771e7d50a17fe80788eeb0f"
Jul 17 14:44:03.773291 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:44:03.773259    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7683-140031008929088\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="99d09cdc-b706-4a52-87e0-041809ee26bf" pod="opendatahub/test-curl-7683-140031008929088"
Jul 17 14:44:04.770400 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:44:04.770368    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7683-140031008929088\" is forbidden: User \"system:node:ip-10-0-137-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-163.ec2.internal' and this object" podUID="99d09cdc-b706-4a52-87e0-041809ee26bf" pod="opendatahub/test-curl-7683-140031008929088"
Jul 17 14:44:04.771967 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:04.771943    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d09cdc-b706-4a52-87e0-041809ee26bf" path="/var/lib/kubelet/pods/99d09cdc-b706-4a52-87e0-041809ee26bf/volumes"
Jul 17 14:44:28.107830 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:28.107797    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"]
Jul 17 14:44:28.108381 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:28.108031    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy" containerID="cri-o://4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1" gracePeriod=30
Jul 17 14:44:31.407215 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:31.407181    2561 patch_prober.go:28] interesting pod/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 14:44:31.407665 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:31.407244    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 14:44:33.368292 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.368233    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:44:33.524024 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.523981    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk6bb\" (UniqueName: \"kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-kube-api-access-kk6bb\") pod \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") "
Jul 17 14:44:33.524211 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524043    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-token\") pod \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") "
Jul 17 14:44:33.524211 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524084    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-socket\") pod \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") "
Jul 17 14:44:33.524211 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524128    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-podinfo\") pod \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") "
Jul 17 14:44:33.524211 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524193    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-credential-socket\") pod \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") "
Jul 17 14:44:33.524428 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524229    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istiod-ca-cert\") pod \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") "
Jul 17 14:44:33.524428 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524260    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-envoy\") pod \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") "
Jul 17 14:44:33.524428 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524301    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-data\") pod \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") "
Jul 17 14:44:33.524428 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524381    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-certs\") pod \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\" (UID: \"99d74494-e49e-4741-9bbb-ed7cdd22b23f\") "
Jul 17 14:44:33.524621 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524420    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99d74494-e49e-4741-9bbb-ed7cdd22b23f" (UID: "99d74494-e49e-4741-9bbb-ed7cdd22b23f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:44:33.524621 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524501    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99d74494-e49e-4741-9bbb-ed7cdd22b23f" (UID: "99d74494-e49e-4741-9bbb-ed7cdd22b23f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:44:33.524767 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524744    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99d74494-e49e-4741-9bbb-ed7cdd22b23f" (UID: "99d74494-e49e-4741-9bbb-ed7cdd22b23f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:44:33.524989 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524966    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:33.525110 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.524994    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:33.525110 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.525032    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:33.525397 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.525230    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99d74494-e49e-4741-9bbb-ed7cdd22b23f" (UID: "99d74494-e49e-4741-9bbb-ed7cdd22b23f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:44:33.525464 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.525421    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-data" (OuterVolumeSpecName: "istio-data") pod "99d74494-e49e-4741-9bbb-ed7cdd22b23f" (UID: "99d74494-e49e-4741-9bbb-ed7cdd22b23f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:44:33.526482 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.526452    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-token" (OuterVolumeSpecName: "istio-token") pod "99d74494-e49e-4741-9bbb-ed7cdd22b23f" (UID: "99d74494-e49e-4741-9bbb-ed7cdd22b23f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:44:33.526727 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.526710    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99d74494-e49e-4741-9bbb-ed7cdd22b23f" (UID: "99d74494-e49e-4741-9bbb-ed7cdd22b23f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:44:33.526820 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.526798    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99d74494-e49e-4741-9bbb-ed7cdd22b23f" (UID: "99d74494-e49e-4741-9bbb-ed7cdd22b23f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:44:33.526877 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.526801    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-kube-api-access-kk6bb" (OuterVolumeSpecName: "kube-api-access-kk6bb") pod "99d74494-e49e-4741-9bbb-ed7cdd22b23f" (UID: "99d74494-e49e-4741-9bbb-ed7cdd22b23f"). InnerVolumeSpecName "kube-api-access-kk6bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:44:33.626096 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.625996    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:33.626096 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.626049    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:33.626096 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.626065    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:33.626096 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.626075    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk6bb\" (UniqueName: \"kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-kube-api-access-kk6bb\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:33.626096 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.626084    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:33.626096 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.626092    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99d74494-e49e-4741-9bbb-ed7cdd22b23f-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:33.933659 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.933624    2561 generic.go:356] "Generic (PLEG): container finished" podID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerID="4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1" exitCode=0
Jul 17 14:44:33.933817 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.933677    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"
Jul 17 14:44:33.933817 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.933676    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" event={"ID":"99d74494-e49e-4741-9bbb-ed7cdd22b23f","Type":"ContainerDied","Data":"4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1"}
Jul 17 14:44:33.933817 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.933801    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2" event={"ID":"99d74494-e49e-4741-9bbb-ed7cdd22b23f","Type":"ContainerDied","Data":"bb04c5631bfc79139f42fc11ea98a9e2d25271b388d17ab3b2e7157b95670d3c"}
Jul 17 14:44:33.933926 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.933831    2561 scope.go:117] "RemoveContainer" containerID="4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1"
Jul 17 14:44:33.943955 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.943936    2561 scope.go:117] "RemoveContainer" containerID="e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215"
Jul 17 14:44:33.952124 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.952108    2561 scope.go:117] "RemoveContainer" containerID="4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1"
Jul 17 14:44:33.952359 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:44:33.952339    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1\": container with ID starting with 4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1 not found: ID does not exist" containerID="4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1"
Jul 17 14:44:33.952440 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.952369    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1"} err="failed to get container status \"4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1\": rpc error: code = NotFound desc = could not find container \"4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1\": container with ID starting with 4d13448a696e71fb0ae3cea3cd2125b9d98e4a26ea8009bb21e3f1c7003108b1 not found: ID does not exist"
Jul 17 14:44:33.952440 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.952391    2561 scope.go:117] "RemoveContainer" containerID="e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215"
Jul 17 14:44:33.952684 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:44:33.952666    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215\": container with ID starting with e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215 not found: ID does not exist" containerID="e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215"
Jul 17 14:44:33.952733 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.952692    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215"} err="failed to get container status \"e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215\": rpc error: code = NotFound desc = could not find container \"e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215\": container with ID starting with e1d6d3d2d06880f12afc77d92ec6456fff0ee5af915c32a922badef2039cf215 not found: ID does not exist"
Jul 17 14:44:33.957826 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.957804    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"]
Jul 17 14:44:33.964421 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:33.964402    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-244a73-openshift-default-f59986d7-hjcj2"]
Jul 17 14:44:34.773577 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:34.773542    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" path="/var/lib/kubelet/pods/99d74494-e49e-4741-9bbb-ed7cdd22b23f/volumes"
Jul 17 14:44:50.753344 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:50.753312    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"]
Jul 17 14:44:50.753817 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:50.753606    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy" containerID="cri-o://502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a" gracePeriod=30
Jul 17 14:44:51.413347 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:51.413314    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fdbc7f8c9-xljlk_43dc360c-549c-4d9b-8dbf-24ca9941c60b/authorino/0.log"
Jul 17 14:44:54.242715 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:54.242683    2561 patch_prober.go:28] interesting pod/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 14:44:54.243085 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:54.242750    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 14:44:56.022990 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.022931    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:44:56.034556 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.034530    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-certs\") pod \"f0e37031-2b99-43ea-8069-f4623447bf26\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") "
Jul 17 14:44:56.034698 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.034587    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcddt\" (UniqueName: \"kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-kube-api-access-vcddt\") pod \"f0e37031-2b99-43ea-8069-f4623447bf26\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") "
Jul 17 14:44:56.034698 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.034620    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e37031-2b99-43ea-8069-f4623447bf26-istio-podinfo\") pod \"f0e37031-2b99-43ea-8069-f4623447bf26\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") "
Jul 17 14:44:56.034830 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.034769    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-socket\") pod \"f0e37031-2b99-43ea-8069-f4623447bf26\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") "
Jul 17 14:44:56.034830 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.034818    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-istio-token\") pod \"f0e37031-2b99-43ea-8069-f4623447bf26\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") "
Jul 17 14:44:56.034940 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.034838    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0e37031-2b99-43ea-8069-f4623447bf26" (UID: "f0e37031-2b99-43ea-8069-f4623447bf26"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:44:56.034940 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.034860    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-data\") pod \"f0e37031-2b99-43ea-8069-f4623447bf26\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") "
Jul 17 14:44:56.034940 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.034912    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-credential-socket\") pod \"f0e37031-2b99-43ea-8069-f4623447bf26\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") "
Jul 17 14:44:56.035123 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.034948    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e37031-2b99-43ea-8069-f4623447bf26-istiod-ca-cert\") pod \"f0e37031-2b99-43ea-8069-f4623447bf26\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") "
Jul 17 14:44:56.035123 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.034993    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-envoy\") pod \"f0e37031-2b99-43ea-8069-f4623447bf26\" (UID: \"f0e37031-2b99-43ea-8069-f4623447bf26\") "
Jul 17 14:44:56.035123 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.035084    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0e37031-2b99-43ea-8069-f4623447bf26" (UID: "f0e37031-2b99-43ea-8069-f4623447bf26"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:44:56.035263 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.035192    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0e37031-2b99-43ea-8069-f4623447bf26" (UID: "f0e37031-2b99-43ea-8069-f4623447bf26"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:44:56.035587 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.035549    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0e37031-2b99-43ea-8069-f4623447bf26-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0e37031-2b99-43ea-8069-f4623447bf26" (UID: "f0e37031-2b99-43ea-8069-f4623447bf26"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:44:56.035708 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.035585    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-credential-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:56.035708 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.035607    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-certs\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:56.035708 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.035622    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-workload-socket\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:56.035859 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.035789    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0e37031-2b99-43ea-8069-f4623447bf26" (UID: "f0e37031-2b99-43ea-8069-f4623447bf26"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:44:56.036918 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.036896    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0e37031-2b99-43ea-8069-f4623447bf26-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0e37031-2b99-43ea-8069-f4623447bf26" (UID: "f0e37031-2b99-43ea-8069-f4623447bf26"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 14:44:56.036918 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.036907    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-kube-api-access-vcddt" (OuterVolumeSpecName: "kube-api-access-vcddt") pod "f0e37031-2b99-43ea-8069-f4623447bf26" (UID: "f0e37031-2b99-43ea-8069-f4623447bf26"). InnerVolumeSpecName "kube-api-access-vcddt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:44:56.037260 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.037240    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0e37031-2b99-43ea-8069-f4623447bf26" (UID: "f0e37031-2b99-43ea-8069-f4623447bf26"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:44:56.037614 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.037588    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0e37031-2b99-43ea-8069-f4623447bf26" (UID: "f0e37031-2b99-43ea-8069-f4623447bf26"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:44:56.070021 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.069982    2561 generic.go:356] "Generic (PLEG): container finished" podID="f0e37031-2b99-43ea-8069-f4623447bf26" containerID="502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a" exitCode=0
Jul 17 14:44:56.070136 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.070037    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" event={"ID":"f0e37031-2b99-43ea-8069-f4623447bf26","Type":"ContainerDied","Data":"502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a"}
Jul 17 14:44:56.070136 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.070071    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"
Jul 17 14:44:56.070136 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.070121    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl" event={"ID":"f0e37031-2b99-43ea-8069-f4623447bf26","Type":"ContainerDied","Data":"8b0ccefb5155afc27bdd0ebd12de05d56c5e4b3f9e279b5372c79b5b237ef74f"}
Jul 17 14:44:56.070247 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.070143    2561 scope.go:117] "RemoveContainer" containerID="502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a"
Jul 17 14:44:56.079598 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.079576    2561 scope.go:117] "RemoveContainer" containerID="ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e"
Jul 17 14:44:56.088914 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.088898    2561 scope.go:117] "RemoveContainer" containerID="502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a"
Jul 17 14:44:56.089207 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:44:56.089188    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a\": container with ID starting with 502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a not found: ID does not exist" containerID="502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a"
Jul 17 14:44:56.089274 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.089216    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a"} err="failed to get container status \"502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a\": rpc error: code = NotFound desc = could not find container \"502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a\": container with ID starting with 502cde787833fa30cef08cd8ac549061e7e4b4ae5feda0f0b9ed282976abc41a not found: ID does not exist"
Jul 17 14:44:56.089274 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.089234    2561 scope.go:117] "RemoveContainer" containerID="ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e"
Jul 17 14:44:56.089474 ip-10-0-137-163 kubenswrapper[2561]: E0717 14:44:56.089456    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e\": container with ID starting with ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e not found: ID does not exist" containerID="ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e"
Jul 17 14:44:56.089529 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.089482    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e"} err="failed to get container status \"ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e\": rpc error: code = NotFound desc = could not find container \"ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e\": container with ID starting with ab4fa7b6ebea52e7d5f035763a6ead79d01a51304725ce41ced4ff9f5422461e not found: ID does not exist"
Jul 17 14:44:56.094456 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.094428    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"]
Jul 17 14:44:56.096467 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.096450    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-9eb655-openshift-default-76775f74c9-qk8gl"]
Jul 17 14:44:56.136887 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.136864    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e37031-2b99-43ea-8069-f4623447bf26-istiod-ca-cert\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:56.136887 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.136886    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-envoy\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:56.136994 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.136894    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcddt\" (UniqueName: \"kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-kube-api-access-vcddt\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:56.136994 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.136903    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e37031-2b99-43ea-8069-f4623447bf26-istio-podinfo\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:56.136994 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.136912    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e37031-2b99-43ea-8069-f4623447bf26-istio-token\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:56.136994 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.136919    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e37031-2b99-43ea-8069-f4623447bf26-istio-data\") on node \"ip-10-0-137-163.ec2.internal\" DevicePath \"\""
Jul 17 14:44:56.772812 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:56.772777    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" path="/var/lib/kubelet/pods/f0e37031-2b99-43ea-8069-f4623447bf26/volumes"
Jul 17 14:44:57.148026 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:57.147936    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hcggw_a500635e-b087-4d94-888b-bada40827251/manager/0.log"
Jul 17 14:44:57.271426 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:57.271400    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-22dhv_0a275428-cc40-4134-9ed6-0f48a8c9d67c/manager/0.log"
Jul 17 14:44:57.530074 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:57.530038    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lhvbm_6b8276bb-9bf9-469e-b299-c4eb405e4b1b/server/0.log"
Jul 17 14:44:57.653586 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:57.653550    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gvxmw_bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6/manager/0.log"
Jul 17 14:44:58.049990 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:58.049967    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57f79678b4-wjnpf_4d74f61d-894d-4405-8f9d-577513bb5489/copy-manifests/0.log"
Jul 17 14:44:58.063984 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:58.063963    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57f79678b4-wjnpf_4d74f61d-894d-4405-8f9d-577513bb5489/manager/0.log"
Jul 17 14:44:58.576586 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:58.576558    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738280-mlw6k_9088857f-996d-4183-8e8c-db56fdf45e8e/cleanup/0.log"
Jul 17 14:44:58.708104 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:44:58.708076    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738295-jqt5q_a38e9089-f0b7-4713-a760-41f11750263f/cleanup/0.log"
Jul 17 14:45:00.303088 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:00.303061    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fdbc7f8c9-xljlk_43dc360c-549c-4d9b-8dbf-24ca9941c60b/authorino/0.log"
Jul 17 14:45:00.428130 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:00.428101    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bgzng_489c3b55-dbf8-4327-90cd-d97b261a24df/manager/0.log"
Jul 17 14:45:00.540939 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:00.540916    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-f5gnj_f9d5b96a-e2a7-4af4-8595-4e517357057a/manager/0.log"
Jul 17 14:45:00.777155 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:00.777119    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-t4mgn_c9d86e80-afd0-40c9-a2cd-52795268301a/registry-server/0.log"
Jul 17 14:45:00.900898 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:00.900873    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-j4qjr_bb136535-9959-49b8-a70e-45ff92f81ef3/manager/0.log"
Jul 17 14:45:01.165638 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:01.165557    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-l6frd_8b0faee6-d205-4fcc-9172-5f70ffb4df39/manager/0.log"
Jul 17 14:45:01.528197 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:01.528173    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ftdhm_b60bfbcc-51ce-4006-b9de-a625cba35cb2/istio-proxy/0.log"
Jul 17 14:45:01.897125 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:01.897043    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66c48b9c48-wv9s2_26f08e15-82c5-4ad5-aafe-8f0a602f98a0/kube-auth-proxy/0.log"
Jul 17 14:45:02.017737 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:02.017707    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vvrbt_3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375/istio-proxy/1.log"
Jul 17 14:45:02.383345 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:02.383315    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-558ff794f6-txt9j_332b1551-2e12-4d8b-8b34-55aeba43b9f3/router/0.log"
Jul 17 14:45:02.862536 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:02.862504    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vr2gm_3908cc41-7d8c-4719-b25f-830e1b08a6f1/main/0.log"
Jul 17 14:45:03.195044 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:03.195016    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k"]
Jul 17 14:45:03.198119 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:03.198096    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738280-mlw6k"]
Jul 17 14:45:04.772080 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:04.772044    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9088857f-996d-4183-8e8c-db56fdf45e8e" path="/var/lib/kubelet/pods/9088857f-996d-4183-8e8c-db56fdf45e8e/volumes"
Jul 17 14:45:09.750042 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.749951    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6g9mf/must-gather-lxjn2"]
Jul 17 14:45:09.750417 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750391    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy"
Jul 17 14:45:09.750417 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750403    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy"
Jul 17 14:45:09.750417 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750411    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy"
Jul 17 14:45:09.750417 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750416    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy"
Jul 17 14:45:09.750575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750425    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy"
Jul 17 14:45:09.750575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750432    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy"
Jul 17 14:45:09.750575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750449    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d09cdc-b706-4a52-87e0-041809ee26bf" containerName="test-curl-7683-140031008929088"
Jul 17 14:45:09.750575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750454    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d09cdc-b706-4a52-87e0-041809ee26bf" containerName="test-curl-7683-140031008929088"
Jul 17 14:45:09.750575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750518    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy"
Jul 17 14:45:09.750575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750525    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy"
Jul 17 14:45:09.750575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750534    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e37031-2b99-43ea-8069-f4623447bf26" containerName="istio-proxy"
Jul 17 14:45:09.750575 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750541    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d09cdc-b706-4a52-87e0-041809ee26bf" containerName="test-curl-7683-140031008929088"
Jul 17 14:45:09.750834 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750599    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy"
Jul 17 14:45:09.750834 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750604    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy"
Jul 17 14:45:09.750834 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.750675    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d74494-e49e-4741-9bbb-ed7cdd22b23f" containerName="istio-proxy"
Jul 17 14:45:09.753863 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.753848    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6g9mf/must-gather-lxjn2"
Jul 17 14:45:09.756525 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.756494    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6g9mf\"/\"kube-root-ca.crt\""
Jul 17 14:45:09.756676 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.756498    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6g9mf\"/\"default-dockercfg-fhjml\""
Jul 17 14:45:09.756676 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.756653    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6g9mf\"/\"openshift-service-ca.crt\""
Jul 17 14:45:09.768388 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.768364    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6g9mf/must-gather-lxjn2"]
Jul 17 14:45:09.859329 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.859304    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a43470b1-9f0f-4f9d-812e-9ee60cef0a9d-must-gather-output\") pod \"must-gather-lxjn2\" (UID: \"a43470b1-9f0f-4f9d-812e-9ee60cef0a9d\") " pod="openshift-must-gather-6g9mf/must-gather-lxjn2"
Jul 17 14:45:09.859448 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.859334    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlvjz\" (UniqueName: \"kubernetes.io/projected/a43470b1-9f0f-4f9d-812e-9ee60cef0a9d-kube-api-access-jlvjz\") pod \"must-gather-lxjn2\" (UID: \"a43470b1-9f0f-4f9d-812e-9ee60cef0a9d\") " pod="openshift-must-gather-6g9mf/must-gather-lxjn2"
Jul 17 14:45:09.960615 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.960589    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a43470b1-9f0f-4f9d-812e-9ee60cef0a9d-must-gather-output\") pod \"must-gather-lxjn2\" (UID: \"a43470b1-9f0f-4f9d-812e-9ee60cef0a9d\") " pod="openshift-must-gather-6g9mf/must-gather-lxjn2"
Jul 17 14:45:09.960736 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.960620    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlvjz\" (UniqueName: \"kubernetes.io/projected/a43470b1-9f0f-4f9d-812e-9ee60cef0a9d-kube-api-access-jlvjz\") pod \"must-gather-lxjn2\" (UID: \"a43470b1-9f0f-4f9d-812e-9ee60cef0a9d\") " pod="openshift-must-gather-6g9mf/must-gather-lxjn2"
Jul 17 14:45:09.960921 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.960903    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a43470b1-9f0f-4f9d-812e-9ee60cef0a9d-must-gather-output\") pod \"must-gather-lxjn2\" (UID: \"a43470b1-9f0f-4f9d-812e-9ee60cef0a9d\") " pod="openshift-must-gather-6g9mf/must-gather-lxjn2"
Jul 17 14:45:09.971320 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:09.971293    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlvjz\" (UniqueName: \"kubernetes.io/projected/a43470b1-9f0f-4f9d-812e-9ee60cef0a9d-kube-api-access-jlvjz\") pod \"must-gather-lxjn2\" (UID: \"a43470b1-9f0f-4f9d-812e-9ee60cef0a9d\") " pod="openshift-must-gather-6g9mf/must-gather-lxjn2"
Jul 17 14:45:10.063172 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:10.063117    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6g9mf/must-gather-lxjn2"
Jul 17 14:45:10.212121 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:10.212090    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6g9mf/must-gather-lxjn2"]
Jul 17 14:45:10.213872 ip-10-0-137-163 kubenswrapper[2561]: W0717 14:45:10.213842    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda43470b1_9f0f_4f9d_812e_9ee60cef0a9d.slice/crio-c2507845c468de5fafb3e7a406fdf1b05bb64e2f725216476d46f1587a82216c WatchSource:0}: Error finding container c2507845c468de5fafb3e7a406fdf1b05bb64e2f725216476d46f1587a82216c: Status 404 returned error can't find the container with id c2507845c468de5fafb3e7a406fdf1b05bb64e2f725216476d46f1587a82216c
Jul 17 14:45:11.157886 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:11.157853    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6g9mf/must-gather-lxjn2" event={"ID":"a43470b1-9f0f-4f9d-812e-9ee60cef0a9d","Type":"ContainerStarted","Data":"1172af8ad4b1c9386490e5a7a88438773ea7f5fca002efe9684ab3a827ffb63c"}
Jul 17 14:45:11.158226 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:11.157907    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6g9mf/must-gather-lxjn2" event={"ID":"a43470b1-9f0f-4f9d-812e-9ee60cef0a9d","Type":"ContainerStarted","Data":"c2507845c468de5fafb3e7a406fdf1b05bb64e2f725216476d46f1587a82216c"}
Jul 17 14:45:12.168069 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:12.168030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6g9mf/must-gather-lxjn2" event={"ID":"a43470b1-9f0f-4f9d-812e-9ee60cef0a9d","Type":"ContainerStarted","Data":"2b39b126b5aeacd36222720ea3eccb69422ff8984b4b327da84fb931ad86d4ca"}
Jul 17 14:45:12.186965 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:12.186909    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6g9mf/must-gather-lxjn2" podStartSLOduration=2.342984934 podStartE2EDuration="3.186889484s" totalImagesPullingTime="843.90455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:45:12.18496067 +0000 UTC m=+3597.891407505" watchObservedRunningTime="2026-07-17 14:45:12.186889484 +0000 UTC m=+3597.893336321"
Jul 17 14:45:12.711821 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:12.711756    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gsr4m_f83860e4-2ac5-4187-aa2f-45fcc803faf5/global-pull-secret-syncer/0.log"
Jul 17 14:45:12.895175 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:12.895141    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dztc6_21b1cb43-5f8c-4bc6-9452-b6f39aa68e71/konnectivity-agent/0.log"
Jul 17 14:45:13.007108 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:13.007034    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-163.ec2.internal_ad89fec431916698e96755d7734948e7/haproxy/0.log"
Jul 17 14:45:15.347984 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:15.347949    2561 scope.go:117] "RemoveContainer" containerID="7cc0647446089ddb4dd2900ae261d9248824dcbfada1067918205d1fdbde4df1"
Jul 17 14:45:18.202950 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:18.202455    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fdbc7f8c9-xljlk_43dc360c-549c-4d9b-8dbf-24ca9941c60b/authorino/0.log"
Jul 17 14:45:18.239461 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:18.237211    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bgzng_489c3b55-dbf8-4327-90cd-d97b261a24df/manager/0.log"
Jul 17 14:45:18.296052 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:18.295984    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-f5gnj_f9d5b96a-e2a7-4af4-8595-4e517357057a/manager/0.log"
Jul 17 14:45:18.404040 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:18.403989    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-t4mgn_c9d86e80-afd0-40c9-a2cd-52795268301a/registry-server/0.log"
Jul 17 14:45:18.572687 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:18.572497    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-j4qjr_bb136535-9959-49b8-a70e-45ff92f81ef3/manager/0.log"
Jul 17 14:45:18.709733 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:18.709689    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-l6frd_8b0faee6-d205-4fcc-9172-5f70ffb4df39/manager/0.log"
Jul 17 14:45:20.420497 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.420468    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e794b55-bc79-438d-863f-a922ce0f6bba/alertmanager/0.log"
Jul 17 14:45:20.447660 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.447632    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e794b55-bc79-438d-863f-a922ce0f6bba/config-reloader/0.log"
Jul 17 14:45:20.472585 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.472559    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e794b55-bc79-438d-863f-a922ce0f6bba/kube-rbac-proxy-web/0.log"
Jul 17 14:45:20.500656 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.500631    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e794b55-bc79-438d-863f-a922ce0f6bba/kube-rbac-proxy/0.log"
Jul 17 14:45:20.532971 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.532933    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e794b55-bc79-438d-863f-a922ce0f6bba/kube-rbac-proxy-metric/0.log"
Jul 17 14:45:20.557971 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.557950    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e794b55-bc79-438d-863f-a922ce0f6bba/prom-label-proxy/0.log"
Jul 17 14:45:20.583353 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.583316    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e794b55-bc79-438d-863f-a922ce0f6bba/init-config-reloader/0.log"
Jul 17 14:45:20.729673 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.729596    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-bdfvl_d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf/kube-state-metrics/0.log"
Jul 17 14:45:20.748892 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.748866    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-bdfvl_d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf/kube-rbac-proxy-main/0.log"
Jul 17 14:45:20.774436 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.774401    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-bdfvl_d802ba2c-8c2f-43f3-b427-6cc29a4b7fcf/kube-rbac-proxy-self/0.log"
Jul 17 14:45:20.806870 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.806836    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-69f8948bd-j5gn8_18bbb18d-fb5b-4a33-856e-11badc504f7b/metrics-server/0.log"
Jul 17 14:45:20.834346 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.834318    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-5nc7j_98716b2a-fe1e-421a-8b56-a4257aaf3bef/monitoring-plugin/0.log"
Jul 17 14:45:20.864255 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.864227    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dhxg7_aa13677c-2c63-4780-8ed8-9de458d822da/node-exporter/0.log"
Jul 17 14:45:20.889451 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.889426    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dhxg7_aa13677c-2c63-4780-8ed8-9de458d822da/kube-rbac-proxy/0.log"
Jul 17 14:45:20.920691 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:20.920666    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dhxg7_aa13677c-2c63-4780-8ed8-9de458d822da/init-textfile/0.log"
Jul 17 14:45:21.449418 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:21.449389    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-nkvs5_637bfa7c-ee64-4429-8726-b0281760e30b/prometheus-operator-admission-webhook/0.log"
Jul 17 14:45:21.484517 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:21.484478    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-794ddcf88c-fmpd8_65051f9e-1a46-40b3-8c44-45437d21f9c8/telemeter-client/0.log"
Jul 17 14:45:21.507629 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:21.507491    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-794ddcf88c-fmpd8_65051f9e-1a46-40b3-8c44-45437d21f9c8/reload/0.log"
Jul 17 14:45:21.530812 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:21.530787    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-794ddcf88c-fmpd8_65051f9e-1a46-40b3-8c44-45437d21f9c8/kube-rbac-proxy/0.log"
Jul 17 14:45:22.670443 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.670404    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"]
Jul 17 14:45:22.678883 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.678855    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.683155 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.683124    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"]
Jul 17 14:45:22.841529 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.841500    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-lib-modules\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.841700 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.841650    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnkq2\" (UniqueName: \"kubernetes.io/projected/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-kube-api-access-nnkq2\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.841743 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.841717    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-proc\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.841788 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.841760    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-sys\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.841878 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.841856    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-podres\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.943374 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.943336    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-proc\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.943806 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.943767    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-sys\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.943930 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.943847    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-podres\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.943930 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.943888    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-lib-modules\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.944086 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.944037    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnkq2\" (UniqueName: \"kubernetes.io/projected/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-kube-api-access-nnkq2\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.944647 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.943682    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-proc\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.944784 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.944651    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-lib-modules\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.944875 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.944695    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-sys\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.944967 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.944805    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-podres\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.960985 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.960949    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnkq2\" (UniqueName: \"kubernetes.io/projected/e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1-kube-api-access-nnkq2\") pod \"perf-node-gather-daemonset-m7v6b\" (UID: \"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1\") " pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:22.996959 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:22.996920    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:23.187574 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:23.184399    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"]
Jul 17 14:45:23.291940 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:23.291912    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b" event={"ID":"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1","Type":"ContainerStarted","Data":"243485e122f2d240f11e496903247786fd9d58a34844f182bd059649c59f0098"}
Jul 17 14:45:23.995373 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:23.995347    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-894c5b5cf-6m2kf_cf24f6a4-cc73-4636-8b5b-e57a18a4f7c1/console/0.log"
Jul 17 14:45:24.031476 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:24.031440    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-w66vp_d38007da-cfed-4ff5-9d29-ce756630ec75/download-server/0.log"
Jul 17 14:45:24.298940 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:24.298848    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b" event={"ID":"e73d110a-e5a5-446c-ab89-f8e0ca0f1dd1","Type":"ContainerStarted","Data":"17c851eae68872a19c6507e84d2117da2fb556138384d061a809b7f79e499c35"}
Jul 17 14:45:24.299142 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:24.299024    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:24.316435 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:24.316392    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b" podStartSLOduration=2.31637635 podStartE2EDuration="2.31637635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:45:24.315516807 +0000 UTC m=+3610.021963644" watchObservedRunningTime="2026-07-17 14:45:24.31637635 +0000 UTC m=+3610.022823183"
Jul 17 14:45:25.719471 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:25.719427    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8r6d5_474b65de-97a7-487f-80f2-0d4c5b3d5df1/dns/0.log"
Jul 17 14:45:25.743399 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:25.743353    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8r6d5_474b65de-97a7-487f-80f2-0d4c5b3d5df1/kube-rbac-proxy/0.log"
Jul 17 14:45:25.881878 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:25.881842    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z77wd_f222c7e6-2c4d-402a-879c-4a0ca6ae2475/dns-node-resolver/0.log"
Jul 17 14:45:26.470618 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:26.470547    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6d879b7597-vdcs5_1c538f37-6861-475b-9681-4190bccfb2d9/registry/0.log"
Jul 17 14:45:26.560343 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:26.560306    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xtx5f_b0825032-81b7-4c8c-8494-cab86102a66c/node-ca/0.log"
Jul 17 14:45:27.579216 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:27.579187    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ftdhm_b60bfbcc-51ce-4006-b9de-a625cba35cb2/istio-proxy/0.log"
Jul 17 14:45:27.761546 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:27.761515    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66c48b9c48-wv9s2_26f08e15-82c5-4ad5-aafe-8f0a602f98a0/kube-auth-proxy/0.log"
Jul 17 14:45:27.792414 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:27.792384    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vvrbt_3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375/istio-proxy/0.log"
Jul 17 14:45:27.828167 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:27.828135    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vvrbt_3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375/istio-proxy/1.log"
Jul 17 14:45:27.917318 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:27.917272    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-558ff794f6-txt9j_332b1551-2e12-4d8b-8b34-55aeba43b9f3/router/0.log"
Jul 17 14:45:28.637164 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:28.637138    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5qnvt_decf2204-ccbc-4915-81f8-009c80ac9398/serve-healthcheck-canary/0.log"
Jul 17 14:45:29.222107 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:29.222073    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-kp8xd_a04aa92c-47f8-4dd2-bc0f-cb8604321dc8/insights-operator/0.log"
Jul 17 14:45:29.222704 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:29.222673    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-kp8xd_a04aa92c-47f8-4dd2-bc0f-cb8604321dc8/insights-operator/1.log"
Jul 17 14:45:29.321353 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:29.321329    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmb5l_dcffd48f-2a19-40a0-a545-ae16a7f763bc/kube-rbac-proxy/0.log"
Jul 17 14:45:29.344916 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:29.344885    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmb5l_dcffd48f-2a19-40a0-a545-ae16a7f763bc/exporter/0.log"
Jul 17 14:45:29.370065 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:29.370031    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmb5l_dcffd48f-2a19-40a0-a545-ae16a7f763bc/extractor/0.log"
Jul 17 14:45:30.319181 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:30.319150    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6g9mf/perf-node-gather-daemonset-m7v6b"
Jul 17 14:45:31.895617 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:31.895585    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hcggw_a500635e-b087-4d94-888b-bada40827251/manager/0.log"
Jul 17 14:45:32.137926 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:32.137768    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-22dhv_0a275428-cc40-4134-9ed6-0f48a8c9d67c/manager/0.log"
Jul 17 14:45:32.271036 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:32.270984    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lhvbm_6b8276bb-9bf9-469e-b299-c4eb405e4b1b/server/0.log"
Jul 17 14:45:32.364214 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:32.364184    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gvxmw_bfe2fe4c-c507-4b75-9e2a-d13ebf1600f6/manager/0.log"
Jul 17 14:45:32.610732 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:32.610704    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57f79678b4-wjnpf_4d74f61d-894d-4405-8f9d-577513bb5489/manager/0.log"
Jul 17 14:45:32.652622 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:32.652589    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57f79678b4-wjnpf_4d74f61d-894d-4405-8f9d-577513bb5489/copy-manifests/0.log"
Jul 17 14:45:33.259337 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:33.259215    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ftdhm_b60bfbcc-51ce-4006-b9de-a625cba35cb2/istio-proxy/0.log"
Jul 17 14:45:33.413646 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:33.413614    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66c48b9c48-wv9s2_26f08e15-82c5-4ad5-aafe-8f0a602f98a0/kube-auth-proxy/0.log"
Jul 17 14:45:33.442094 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:33.442068    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vvrbt_3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375/istio-proxy/0.log"
Jul 17 14:45:33.484553 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:33.484516    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vvrbt_3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375/istio-proxy/1.log"
Jul 17 14:45:33.540747 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:33.540668    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-558ff794f6-txt9j_332b1551-2e12-4d8b-8b34-55aeba43b9f3/router/0.log"
Jul 17 14:45:34.079555 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:34.079530    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-bsl84_25d89ea4-20ad-4571-b01c-5a3d727cefc7/openshift-lws-operator/0.log"
Jul 17 14:45:34.834401 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:34.834372    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ftdhm_b60bfbcc-51ce-4006-b9de-a625cba35cb2/istio-proxy/0.log"
Jul 17 14:45:34.987224 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:34.987197    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66c48b9c48-wv9s2_26f08e15-82c5-4ad5-aafe-8f0a602f98a0/kube-auth-proxy/0.log"
Jul 17 14:45:35.008750 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:35.008692    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vvrbt_3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375/istio-proxy/0.log"
Jul 17 14:45:35.044995 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:35.044960    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vvrbt_3bb5bcb8-d6f7-4b9a-8b6c-70f159d2a375/istio-proxy/1.log"
Jul 17 14:45:35.104421 ip-10-0-137-163 kubenswrapper[2561]: I0717 14:45:35.104339    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-558ff794f6-txt9j_332b1551-2e12-4d8b-8b34-55aeba43b9f3/router/0.log"