Jul 27 17:21:02.164055 ip-10-0-133-68 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 17:21:02.570851 ip-10-0-133-68 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:21:02.570851 ip-10-0-133-68 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 17:21:02.570851 ip-10-0-133-68 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:21:02.570851 ip-10-0-133-68 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 17:21:02.570851 ip-10-0-133-68 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:21:02.572806 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.572665    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 17:21:02.577614 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577576    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:21:02.577614 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577608    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:21:02.577614 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577613    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:21:02.577614 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577617    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:21:02.577614 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577621    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577626    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577633    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577638    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577642    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577646    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577651    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577654    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577658    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577662    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577665    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577670    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577673    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577677    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577699    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577705    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577709    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577713    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577717    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:21:02.577996 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577721    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577725    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577729    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577733    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577737    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577741    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577754    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577758    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577762    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577765    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577770    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577773    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577777    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577783    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577788    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577793    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577798    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577802    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577806    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577810    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:21:02.578784 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577813    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577818    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577824    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577828    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577832    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577837    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577841    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577847    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577851    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577855    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577860    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577864    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577868    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577872    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577876    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577881    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577886    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577890    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577894    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577899    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:21:02.579672 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577903    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577907    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577912    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577916    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577920    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577924    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577929    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577934    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577939    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577943    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577948    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577952    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577956    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577961    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577965    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577969    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577974    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577978    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577982    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577986    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:21:02.580610 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577991    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577995    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.577999    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578005    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578010    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578014    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578018    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578024    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578031    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578036    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578041    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578046    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578051    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578055    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578061    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.578065    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579294    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579306    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579310    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:21:02.581491 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579319    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579325    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579330    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579335    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579339    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579345    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579350    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579354    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579359    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579363    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579367    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579372    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579376    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579380    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579386    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579390    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579395    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579399    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579404    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579408    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:21:02.582323 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579412    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579416    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579420    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579425    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579429    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579433    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579437    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579441    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579446    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579453    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579458    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579462    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579467    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579471    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579476    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579480    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579484    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579489    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579493    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579497    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:21:02.582975 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579502    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579506    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579510    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579516    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579520    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579525    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579529    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579536    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579540    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579545    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579549    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579553    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579558    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579562    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579567    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579572    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579575    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579580    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579584    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:21:02.583583 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579589    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579593    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579597    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579601    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579605    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579611    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579615    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579620    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579624    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579628    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579632    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579637    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579641    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579645    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579650    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579657    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579661    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579665    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579669    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579673    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:21:02.584258 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579677    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579705    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579711    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579715    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579719    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579723    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579728    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579732    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579736    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579741    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579745    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579749    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579754    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579761    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579766    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579771    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.579776    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580425    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580433    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:21:02.584860 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580439    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580446    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580450    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580455    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580460    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580464    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580469    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580473    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580477    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580481    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580485    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580489    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580492    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580496    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580502    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580506    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580510    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580515    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580519    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:21:02.585356 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580523    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580527    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580531    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580536    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580540    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580545    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580549    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580553    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580557    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580561    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580565    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580569    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580573    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580577    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580581    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580585    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580589    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580594    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580598    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580602    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:21:02.585929 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580608    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580612    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580616    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580620    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580625    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580629    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580633    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580637    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580648    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580653    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580657    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580662    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580667    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580671    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580676    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580704    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580709    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580713    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580719    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:21:02.586419 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580723    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580730    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580736    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580741    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580745    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580751    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580756    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580761    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580766    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580770    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580775    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580779    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580786    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580791    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580795    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580799    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580803    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580808    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580812    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:21:02.586961 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580816    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580821    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580825    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580830    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580834    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580842    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580846    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580851    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580855    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580860    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580864    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580869    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580875    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580880    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580884    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580888    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580893    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580897    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580901    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.580906    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:21:02.587416 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581016    2584 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581029    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581039    2584 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581046    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581066    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581071    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581078    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581085    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581091    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581096    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581102    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581107    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581112    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581117    2584 flags.go:64] FLAG: --cgroup-root=""
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581122    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581127    2584 flags.go:64] FLAG: --client-ca-file=""
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581133    2584 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581140    2584 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581148    2584 flags.go:64] FLAG: --cluster-domain=""
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581152    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581157    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581162    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581168    2584 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581175    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 17:21:02.587931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581180    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581185    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581190    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581195    2584 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581200    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581204    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581209    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581214    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581221    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581226    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581231    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581236    2584 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581240    2584 flags.go:64] FLAG: --enable-server="true"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581245    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581252    2584 flags.go:64] FLAG: --event-burst="100"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581258    2584 flags.go:64] FLAG: --event-qps="50"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581262    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581267    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581273    2584 flags.go:64] FLAG: --eviction-hard=""
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581279    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581283    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581288    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581293    2584 flags.go:64] FLAG: --eviction-soft=""
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581298    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581303    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 17:21:02.588555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581311    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581316    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581320    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581325    2584 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581330    2584 flags.go:64] FLAG: --feature-gates=""
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581336    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581341    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581346    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581351    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581356    2584 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581361    2584 flags.go:64] FLAG: --help="false"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581366    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-133-68.ec2.internal"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581371    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581376    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581381    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581386    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581392    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581397    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581401    2584 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581406    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581410    2584 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581416    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581422    2584 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581426    2584 flags.go:64] FLAG: --kube-reserved=""
Jul 27 17:21:02.589235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581431    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581436    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581441    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581446    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581450    2584 flags.go:64] FLAG: --lock-file=""
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581455    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581461    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581466    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581476    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581482    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581487    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581492    2584 flags.go:64] FLAG: --logging-format="text"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581496    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581501    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581506    2584 flags.go:64] FLAG: --manifest-url=""
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581510    2584 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581517    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581522    2584 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581528    2584 flags.go:64] FLAG: --max-pods="110"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581533    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581538    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581542    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581547    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581552    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581557    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 17:21:02.589859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581562    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581574    2584 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581580    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581585    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581590    2584 flags.go:64] FLAG: --pod-cidr=""
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581595    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581603    2584 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581608    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581613    2584 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581618    2584 flags.go:64] FLAG: --port="10250"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581623    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581627    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ff61f0dde4dc8f39"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581633    2584 flags.go:64] FLAG: --qos-reserved=""
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581638    2584 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581642    2584 flags.go:64] FLAG: --register-node="true"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581647    2584 flags.go:64] FLAG: --register-with-taints=""
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581653    2584 flags.go:64] FLAG: --registry-burst="10"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581659    2584 flags.go:64] FLAG: --registry-qps="5"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581664    2584 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581671    2584 flags.go:64] FLAG: --reserved-memory=""
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581678    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581701    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581707    2584 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581712    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581716    2584 flags.go:64] FLAG: --runonce="false"
Jul 27 17:21:02.590483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581720    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581726    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581731    2584 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581735    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581740    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581745    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581750    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581755    2584 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581760    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581765    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581770    2584 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581775    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581780    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581785    2584 flags.go:64] FLAG: --system-cgroups=""
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581790    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581800    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581805    2584 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581809    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581819    2584 flags.go:64] FLAG: --tls-min-version=""
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581824    2584 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581829    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581834    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581838    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581844    2584 flags.go:64] FLAG: --v="2"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581852    2584 flags.go:64] FLAG: --version="false"
Jul 27 17:21:02.591213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581859    2584 flags.go:64] FLAG: --vmodule=""
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581867    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.581873    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582075    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582083    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582088    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582093    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582098    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582102    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582108    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582112    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582117    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582121    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582126    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582130    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582135    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582139    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582143    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582149    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582153    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:21:02.591892 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582158    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582162    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582166    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582170    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582175    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582179    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582183    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582187    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582191    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582195    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582199    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582204    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582208    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582221    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582226    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582230    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582237    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582243    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582248    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:21:02.592381 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582252    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582257    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582263    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582269    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582274    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582279    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582284    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582288    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582293    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582298    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582302    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582306    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582310    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582314    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582319    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582322    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582326    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582331    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582335    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582338    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:21:02.592870 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582342    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582346    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582350    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582355    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582359    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582363    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582369    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582381    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582386    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582390    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582394    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582398    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582402    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582407    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582411    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582415    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582419    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582423    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582428    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582432    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:21:02.593360 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582436    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582440    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582444    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582448    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582453    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582457    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582461    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582465    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582469    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582473    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582477    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582481    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582485    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582492    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582497    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582504    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582509    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582513    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582520    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:21:02.593876 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582525    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582530    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582534    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.582538    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.583182    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.590908    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.590931    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.591743    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.591751    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591815    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591823    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591826    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591830    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591833    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:21:02.594396 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591836    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591839    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591842    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591844    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591847    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591850    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591854    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591858    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591861    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591864    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591866    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591869    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591872    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591875    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591877    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591880    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591883    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591885    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591888    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:21:02.594880 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591892    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591895    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591898    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591901    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591904    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591907    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591910    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591913    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591915    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591918    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591920    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591923    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591926    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591928    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591931    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591933    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591936    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591938    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591941    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591943    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:21:02.595342 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591946    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591948    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591951    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591953    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591956    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591958    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591961    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591965    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591969    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591971    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591974    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591977    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591981    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591984    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591987    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591990    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591992    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591995    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.591998    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:21:02.595851 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592001    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592003    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592008    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592011    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592014    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592017    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592020    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592023    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592025    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592028    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592031    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592033    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592036    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592038    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592041    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592043    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592046    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592048    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592051    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592053    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:21:02.596321 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592056    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592058    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592061    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592063    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592066    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592069    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592071    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592075    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592079    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592082    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592085    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592087    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592090    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592092    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592094    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:21:02.596830 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592097    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.592102    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592251    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592257    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592260    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592263    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592265    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592268    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592270    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592273    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592275    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592278    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592282    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592286    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:21:02.597196 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592289    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592292    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592295    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592298    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592301    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592304    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592307    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592310    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592312    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592315    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592319    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592322    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592325    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592329    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592331    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592335    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592338    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592340    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592343    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592345    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:21:02.597544 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592348    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592351    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592355    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592358    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592360    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592362    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592365    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592367    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592370    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592373    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592375    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592377    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592380    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592384    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592388    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592391    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592394    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592397    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592400    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592403    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:21:02.598047 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592406    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592409    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592412    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592414    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592418    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592421    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592423    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592426    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592428    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592430    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592433    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592435    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592438    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592440    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592443    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592445    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592448    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592450    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592453    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592455    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:21:02.598527 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592457    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592460    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592462    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592465    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592467    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592470    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592473    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592476    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592479    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592481    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592484    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592486    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592489    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592491    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592494    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592496    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592499    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592501    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592504    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:21:02.599088 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592507    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592509    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592512    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592514    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592517    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592519    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592522    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:02.592524    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.592529    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.592667    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.596959    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.598536    2584 server.go:1019] "Starting client certificate rotation"
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.598642    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 17:21:02.599554 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.598679    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 17:21:02.601181 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.601164    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 17:21:02.604977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.604952    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 17:21:02.613395 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.613357    2584 log.go:25] "Validated CRI v1 runtime API"
Jul 27 17:21:02.616108 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.616086    2584 log.go:25] "Validated CRI v1 image API"
Jul 27 17:21:02.617013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.616985    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 17:21:02.620017 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.619990    2584 fs.go:135] Filesystem UUIDs: map[1c2430dd-9cee-4625-bc3f-38ffc41d95d2:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 935b3b31-04fb-44fe-864f-fd861603333f:/dev/nvme0n1p3]
Jul 27 17:21:02.620089 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.620018    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 17:21:02.625344 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.625215    2584 manager.go:216] Machine: {Timestamp:2026-07-27 17:21:02.623912119 +0000 UTC m=+0.332060428 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3081027 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21ca42d5eaee37f79ee8f8026ffcc6 SystemUUID:ec21ca42-d5ea-ee37-f79e-e8f8026ffcc6 BootID:883e947f-dd25-4f5c-b461-6c505ef1f9e4 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:64:04:f7:4b:2b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:64:04:f7:4b:2b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:ee:51:2f:c9:57 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 17:21:02.625344 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.625339    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 17:21:02.625458 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.625436    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 17:21:02.627105 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.627087    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 17:21:02.628362 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.628334    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 17:21:02.628512 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.628367    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-68.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 17:21:02.629102 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.629089    2584 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 17:21:02.629144 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.629106    2584 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 17:21:02.629144 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.629114    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 17:21:02.629772 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.629760    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 17:21:02.630518 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.630506    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 17:21:02.632510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.632496    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 17:21:02.632660 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.632650    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 17:21:02.632710 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.632697    2584 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 17:21:02.632710 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.632708    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 17:21:02.632778 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.632724    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 17:21:02.632778 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.632737    2584 kubelet.go:398] "Adding apiserver pod source"
Jul 27 17:21:02.632778 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.632753    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 17:21:02.633166 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.633152    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 17:21:02.633232 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.633180    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 17:21:02.635582 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.635565    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 17:21:02.637467 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.637451    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 17:21:02.637527 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.637498    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 17:21:02.638959 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.638946    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 17:21:02.639009 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.638966    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 17:21:02.639009 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.638972    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 17:21:02.639009 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.638982    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 17:21:02.639009 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.638995    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 17:21:02.639009 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.639003    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 17:21:02.639164 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.639018    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 17:21:02.639164 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.639024    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 17:21:02.639164 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.639031    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 17:21:02.639164 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.639037    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 17:21:02.639164 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.639049    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 17:21:02.639164 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.639055    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 17:21:02.639896 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.639883    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 17:21:02.639896 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.639896    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 17:21:02.641470 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.641447    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-68.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 17:21:02.641638 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.641606    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 17:21:02.641714 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.641654    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 17:21:02.643538 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.643524    2584 server.go:1268] "Started kubelet"
Jul 27 17:21:02.643634 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.643605    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 17:21:02.643674 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.643606    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 17:21:02.644811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.644781    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 17:21:02.644953 ip-10-0-133-68 systemd[1]: Started Kubernetes Kubelet.
Jul 27 17:21:02.646629 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.646611    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 17:21:02.647636 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.647615    2584 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 17:21:02.648302 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.648281    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qh52l"
Jul 27 17:21:02.652337 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.652316    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 17:21:02.653656 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.653642    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 17:21:02.653725 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.653647    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 17:21:02.653788 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.653756    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:02.654268 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654250    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 17:21:02.654268 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654268    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 17:21:02.654412 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654249    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 17:21:02.654412 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654391    2584 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 17:21:02.654412 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654400    2584 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 17:21:02.654627 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654608    2584 factory.go:55] Registering systemd factory
Jul 27 17:21:02.654627 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654629    2584 factory.go:223] Registration of the systemd container factory successfully
Jul 27 17:21:02.654901 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654884    2584 factory.go:153] Registering CRI-O factory
Jul 27 17:21:02.654901 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654902    2584 factory.go:223] Registration of the crio container factory successfully
Jul 27 17:21:02.655035 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654959    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 17:21:02.655035 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.654989    2584 factory.go:103] Registering Raw factory
Jul 27 17:21:02.655035 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.655006    2584 manager.go:1196] Started watching for new ooms in manager
Jul 27 17:21:02.656094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.656076    2584 manager.go:349] Starting recovery of all containers
Jul 27 17:21:02.657206 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.657185    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qh52l"
Jul 27 17:21:02.660427 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.660400    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 17:21:02.660522 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.660502    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 17:21:02.661529 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.660423    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-68.ec2.internal.18c63542a4883e1f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-68.ec2.internal,UID:ip-10-0-133-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-68.ec2.internal,},FirstTimestamp:2026-07-27 17:21:02.643486239 +0000 UTC m=+0.351634655,LastTimestamp:2026-07-27 17:21:02.643486239 +0000 UTC m=+0.351634655,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-68.ec2.internal,}"
Jul 27 17:21:02.663977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.663946    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 17:21:02.666580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.666562    2584 manager.go:354] Recovery completed
Jul 27 17:21:02.668145 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.668114    2584 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 27 17:21:02.671460 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.671442    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:21:02.673369 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.673352    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:21:02.673417 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.673383    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:21:02.673417 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.673399    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:21:02.674167 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.674153    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 17:21:02.674167 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.674160    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:21:02.674167 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.674167    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 17:21:02.674341 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.674185    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 17:21:02.675045 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.675030    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:21:02.675097 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.675072    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:21:02.675097 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.675084    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:21:02.675255 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.675185    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-68.ec2.internal.18c63542a6503c4a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-68.ec2.internal,UID:ip-10-0-133-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-68.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-68.ec2.internal,},FirstTimestamp:2026-07-27 17:21:02.673370186 +0000 UTC m=+0.381518495,LastTimestamp:2026-07-27 17:21:02.673370186 +0000 UTC m=+0.381518495,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-68.ec2.internal,}"
Jul 27 17:21:02.675811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.675798    2584 policy_none.go:49] "None policy: Start"
Jul 27 17:21:02.675864 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.675816    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 17:21:02.675864 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.675826    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 17:21:02.676904 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.676894    2584 policy_none.go:47] "Start"
Jul 27 17:21:02.715597 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.715418    2584 manager.go:337] "Starting Device Plugin manager"
Jul 27 17:21:02.715752 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.715641    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 17:21:02.715752 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.715658    2584 server.go:87] "Starting device plugin registration server"
Jul 27 17:21:02.715948 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.715935    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 17:21:02.715983 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.715950    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 17:21:02.716084 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.716060    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 17:21:02.716260 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.716165    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 17:21:02.716260 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.716177    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 17:21:02.716817 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.716796    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 17:21:02.716905 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.716857    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:02.816498 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.816470    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:21:02.819363 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.816468    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 17:21:02.819363 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.816587    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 17:21:02.819363 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.816616    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 17:21:02.819363 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.816662    2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 17:21:02.819363 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.817482    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:21:02.819363 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.817515    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:21:02.819363 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.817527    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:21:02.819363 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.817553    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-68.ec2.internal"
Jul 27 17:21:02.819363 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.818835    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:21:02.822938 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.822878    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-68.ec2.internal"
Jul 27 17:21:02.822938 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.822904    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-68.ec2.internal\": node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:02.871885 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.871848    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:02.917056 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.917004    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal"]
Jul 27 17:21:02.917233 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.917113    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:21:02.918133 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.918106    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:21:02.918271 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.918144    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:21:02.918271 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.918170    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:21:02.919438 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.919421    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:21:02.919589 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.919572    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:02.919639 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.919605    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:21:02.920229 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.920211    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:21:02.920285 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.920241    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:21:02.920285 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.920251    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:21:02.920783 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.920766    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:21:02.920839 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.920800    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:21:02.920839 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.920818    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:21:02.921784 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.921770    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:02.921830 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.921796    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:21:02.922525 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.922511    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:21:02.922585 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.922536    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:21:02.922585 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.922546    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:21:02.946767 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.946745    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-68.ec2.internal\" not found" node="ip-10-0-133-68.ec2.internal"
Jul 27 17:21:02.950442 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.950425    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-68.ec2.internal\" not found" node="ip-10-0-133-68.ec2.internal"
Jul 27 17:21:02.956952 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.956926    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/652ec2025569d089187b9f40ad489f57-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal\" (UID: \"652ec2025569d089187b9f40ad489f57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:02.956952 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.956955    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652ec2025569d089187b9f40ad489f57-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal\" (UID: \"652ec2025569d089187b9f40ad489f57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:02.957118 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:02.956973    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47b82ae00a4e61d8c8286ec65e9c9c5f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-68.ec2.internal\" (UID: \"47b82ae00a4e61d8c8286ec65e9c9c5f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:02.972593 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:02.972565    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:03.057831 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.057803    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/652ec2025569d089187b9f40ad489f57-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal\" (UID: \"652ec2025569d089187b9f40ad489f57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:03.057831 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.057836    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652ec2025569d089187b9f40ad489f57-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal\" (UID: \"652ec2025569d089187b9f40ad489f57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:03.058015 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.057856    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47b82ae00a4e61d8c8286ec65e9c9c5f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-68.ec2.internal\" (UID: \"47b82ae00a4e61d8c8286ec65e9c9c5f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:03.058015 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.057905    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652ec2025569d089187b9f40ad489f57-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal\" (UID: \"652ec2025569d089187b9f40ad489f57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:03.058015 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.057915    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/652ec2025569d089187b9f40ad489f57-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal\" (UID: \"652ec2025569d089187b9f40ad489f57\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:03.058015 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.058000    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47b82ae00a4e61d8c8286ec65e9c9c5f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-68.ec2.internal\" (UID: \"47b82ae00a4e61d8c8286ec65e9c9c5f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:03.072992 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:03.072914    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:03.173848 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:03.173803    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:03.254551 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.254510    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:03.254722 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.254510    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:03.274798 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:03.274762    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:03.375370 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:03.375267    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:03.476040 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:03.475996    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:03.577173 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:03.577123    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:03.598967 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.598928    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 17:21:03.599174 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.599154    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 17:21:03.663819 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.663715    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 17:16:03 +0000 UTC" deadline="2028-05-05 17:11:37.118472242 +0000 UTC"
Jul 27 17:21:03.663819 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.663752    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15551h50m33.454724279s"
Jul 27 17:21:03.678032 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:03.678002    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:03.678217 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.678067    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:21:03.778715 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:03.778672    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-68.ec2.internal\" not found"
Jul 27 17:21:03.791281 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.791250    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:21:03.824159 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:03.824122    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652ec2025569d089187b9f40ad489f57.slice/crio-e0b0e34b34df824283df0dc9764a35277a583cebb2628e552b41750b2e5e8264 WatchSource:0}: Error finding container e0b0e34b34df824283df0dc9764a35277a583cebb2628e552b41750b2e5e8264: Status 404 returned error can't find the container with id e0b0e34b34df824283df0dc9764a35277a583cebb2628e552b41750b2e5e8264
Jul 27 17:21:03.824522 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:03.824492    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b82ae00a4e61d8c8286ec65e9c9c5f.slice/crio-dba05995f16ea4daffc24a9ab2c028891cb3ce1bdc120dbeedd7d6ddbf412c8b WatchSource:0}: Error finding container dba05995f16ea4daffc24a9ab2c028891cb3ce1bdc120dbeedd7d6ddbf412c8b: Status 404 returned error can't find the container with id dba05995f16ea4daffc24a9ab2c028891cb3ce1bdc120dbeedd7d6ddbf412c8b
Jul 27 17:21:03.830861 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.830838    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:21:03.846046 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.846015    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:21:03.853985 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.853949    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:03.864300 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.864275    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 17:21:03.865365 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.865349    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal"
Jul 27 17:21:03.876788 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:03.876767    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 17:21:04.633818 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.633782    2584 apiserver.go:52] "Watching apiserver"
Jul 27 17:21:04.644990 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.644957    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 17:21:04.645472 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.645439    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-bfl69","openshift-multus/network-metrics-daemon-xk28w","openshift-network-operator/iptables-alerter-94rt8","openshift-ovn-kubernetes/ovnkube-node-dkxjp","kube-system/konnectivity-agent-hvmch","kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal","openshift-dns/node-resolver-vx8bd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal","openshift-multus/multus-additional-cni-plugins-kbt4b","openshift-network-diagnostics/network-check-target-r8f9c","kube-system/global-pull-secret-syncer-lc6cl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg","openshift-cluster-node-tuning-operator/tuned-sgdhl","openshift-image-registry/node-ca-ljv28"]
Jul 27 17:21:04.647948 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.647579    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.649459 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.648934    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.651573 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.650638    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:04.651573 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.650741    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pjdbj\""
Jul 27 17:21:04.651573 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.651171    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 17:21:04.651573 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.651316    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 17:21:04.652788 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.652469    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:21:04.652788 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.652499    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.652788 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.652560    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xp4lh\""
Jul 27 17:21:04.653971 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.653951    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 17:21:04.654655 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.654638    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:04.655169 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.655147    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 17:21:04.655297 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.655264    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 17:21:04.655418 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.655321    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:21:04.655539 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.655523    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:04.655606 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.655588    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tgcgh\""
Jul 27 17:21:04.655669 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.655654    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 17:21:04.656776 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.656737    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.657220 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.657197    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 17:21:04.657621 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.657422    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 17:21:04.657621 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.657452    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 17:21:04.657621 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.657486    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 17:21:04.657621 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.657498    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qg782\""
Jul 27 17:21:04.657621 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.657585    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 17:21:04.657621 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.657589    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 17:21:04.658357 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.658330    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 17:21:04.658466 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.658448    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.659115 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.659023    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 17:21:04.659723 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.659703    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 17:21:04.659993 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.659972    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9j5fv\""
Jul 27 17:21:04.660075 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.660006    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:04.660251 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.660229    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 17:21:04.660251 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.660222    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:04.660487 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.660391    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pxtmn\""
Jul 27 17:21:04.660487 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.660466    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vl4wk\""
Jul 27 17:21:04.660746 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.660729    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 17:21:04.660877 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.660830    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 17:21:04.660877 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.660875    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 17:21:04.661110 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.661092    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 17:21:04.661233 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.661191    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 17:21:04.661233 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.661198    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 17:21:04.661394 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.661317    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 17:21:04.661551 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.661531    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wpb8l\""
Jul 27 17:21:04.661928 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.661910    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:04.662023 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.661965    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:04.663432 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.663411    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:04.663528 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.663484    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:04.665536 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.665113    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.666752 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.666729    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-run\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.666870 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.666780    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-cni-bin\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.666870 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.666806    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-cni-netd\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.666870 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.666822    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1d27ec4-585a-4bbb-820c-315e02774e2b-env-overrides\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.666870 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.666838    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee67f1bd-8263-43a0-b0fd-d552419d88a6-agent-certs\") pod \"konnectivity-agent-hvmch\" (UID: \"ee67f1bd-8263-43a0-b0fd-d552419d88a6\") " pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:04.667074 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.666878    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f02fbba4-deae-4855-8e20-88163660aa6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.667074 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.666907    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-host\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.667074 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.666932    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-etc-openvswitch\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.667224 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.667168    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1d27ec4-585a-4bbb-820c-315e02774e2b-ovnkube-config\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.667224 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.667215    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1d27ec4-585a-4bbb-820c-315e02774e2b-ovnkube-script-lib\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.667331 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.667248    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee67f1bd-8263-43a0-b0fd-d552419d88a6-konnectivity-ca\") pod \"konnectivity-agent-hvmch\" (UID: \"ee67f1bd-8263-43a0-b0fd-d552419d88a6\") " pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:04.667331 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.667287    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-sysconfig\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.667331 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.667323    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-sysctl-d\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.667542 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.667357    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-var-lib-kubelet\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.667603 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.667575    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88de4096-fcfd-4023-9826-b3cc4301b195-iptables-alerter-script\") pod \"iptables-alerter-94rt8\" (UID: \"88de4096-fcfd-4023-9826-b3cc4301b195\") " pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:04.667663 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.667606    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-kubelet\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.667663 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.667645    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-node-log\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.673632 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.667696    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.673886 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.673865    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jglpj\" (UniqueName: \"kubernetes.io/projected/f1d27ec4-585a-4bbb-820c-315e02774e2b-kube-api-access-jglpj\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.674069 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.674051    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjtxn\" (UniqueName: \"kubernetes.io/projected/88de4096-fcfd-4023-9826-b3cc4301b195-kube-api-access-wjtxn\") pod \"iptables-alerter-94rt8\" (UID: \"88de4096-fcfd-4023-9826-b3cc4301b195\") " pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:04.674191 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.674176    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-slash\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.674340 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.674283    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-log-socket\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.674455 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.674440    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb5hs\" (UniqueName: \"kubernetes.io/projected/3c4df1df-58f8-48cf-b5cc-2ca9669d7f32-kube-api-access-gb5hs\") pod \"node-ca-ljv28\" (UID: \"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32\") " pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:04.674603 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.674587    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-cnibin\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.674750 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.674695    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f02fbba4-deae-4855-8e20-88163660aa6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.674899 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.674881    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.675054 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.675038    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f02fbba4-deae-4855-8e20-88163660aa6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.675248 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.675187    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-systemd\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.675361 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.675346    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8428800c-84eb-4d79-a6f2-313e1ae6ad23-tmp\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.675532 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.675514    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-run-netns\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.675649 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.675633    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-run-systemd\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.676033 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.675959    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d4nc\" (UniqueName: \"kubernetes.io/projected/54626d74-62d9-4fea-ae0a-1a73a8f7fee3-kube-api-access-7d4nc\") pod \"node-resolver-vx8bd\" (UID: \"54626d74-62d9-4fea-ae0a-1a73a8f7fee3\") " pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.676127 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676051    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-sys\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.676127 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676076    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-systemd-units\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.676240 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676123    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-var-lib-openvswitch\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.676240 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676155    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-run-openvswitch\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.676240 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676234    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-run-ovn\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.676389 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676268    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54626d74-62d9-4fea-ae0a-1a73a8f7fee3-hosts-file\") pod \"node-resolver-vx8bd\" (UID: \"54626d74-62d9-4fea-ae0a-1a73a8f7fee3\") " pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.676389 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676321    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-modprobe-d\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.676389 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676356    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-tuned\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.676537 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676390    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88de4096-fcfd-4023-9826-b3cc4301b195-host-slash\") pod \"iptables-alerter-94rt8\" (UID: \"88de4096-fcfd-4023-9826-b3cc4301b195\") " pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:04.676537 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676468    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-run-ovn-kubernetes\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.676537 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676500    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1d27ec4-585a-4bbb-820c-315e02774e2b-ovn-node-metrics-cert\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.676537 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676533    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3c4df1df-58f8-48cf-b5cc-2ca9669d7f32-serviceca\") pod \"node-ca-ljv28\" (UID: \"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32\") " pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:04.676761 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676565    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54626d74-62d9-4fea-ae0a-1a73a8f7fee3-tmp-dir\") pod \"node-resolver-vx8bd\" (UID: \"54626d74-62d9-4fea-ae0a-1a73a8f7fee3\") " pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.676761 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676645    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-lib-modules\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.676761 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676679    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw6nm\" (UniqueName: \"kubernetes.io/projected/8428800c-84eb-4d79-a6f2-313e1ae6ad23-kube-api-access-rw6nm\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.676761 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676709    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c4df1df-58f8-48cf-b5cc-2ca9669d7f32-host\") pod \"node-ca-ljv28\" (UID: \"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32\") " pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:04.676761 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676741    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-system-cni-dir\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.677013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676841    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-os-release\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.677013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676879    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcbxb\" (UniqueName: \"kubernetes.io/projected/f02fbba4-deae-4855-8e20-88163660aa6e-kube-api-access-lcbxb\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.677013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676914    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-kubernetes\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.677013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.676952    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-sysctl-conf\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.677786 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.677738    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 17:21:04.678102 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.678063    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d6chq\""
Jul 27 17:21:04.678429 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.678375    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 17:21:04.678843 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.678674    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 17:21:04.685311 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.685281    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 17:21:04.685443 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.685329    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:21:04.708848 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.708620    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-54bs5"
Jul 27 17:21:04.718060 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.718029    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-54bs5"
Jul 27 17:21:04.755833 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.755806    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 17:21:04.777921 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.777886    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1d27ec4-585a-4bbb-820c-315e02774e2b-ovnkube-script-lib\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.778104 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.777921    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-var-lib-cni-multus\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.778104 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.777960    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f02fbba4-deae-4855-8e20-88163660aa6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.778104 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.777986    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-sysconfig\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.778104 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778009    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-sysctl-d\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.778256 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778119    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-sysconfig\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.778256 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778170    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-var-lib-kubelet\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.778256 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778186    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-sysctl-d\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.778256 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778213    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-kubelet\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.778256 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778238    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-node-log\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.778256 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778243    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-var-lib-kubelet\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.778468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778267    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb5hs\" (UniqueName: \"kubernetes.io/projected/3c4df1df-58f8-48cf-b5cc-2ca9669d7f32-kube-api-access-gb5hs\") pod \"node-ca-ljv28\" (UID: \"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32\") " pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:04.778468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778294    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-kubelet\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.778468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778300    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f02fbba4-deae-4855-8e20-88163660aa6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.778468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778313    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-node-log\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.778468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778330    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjtxn\" (UniqueName: \"kubernetes.io/projected/88de4096-fcfd-4023-9826-b3cc4301b195-kube-api-access-wjtxn\") pod \"iptables-alerter-94rt8\" (UID: \"88de4096-fcfd-4023-9826-b3cc4301b195\") " pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:04.778468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778360    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-socket-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.778468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778388    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-system-cni-dir\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.778468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778411    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-var-lib-cni-bin\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.778468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778444    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-hostroot\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778469    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/968fd454-81ec-4346-85a0-967a08e76ee6-multus-daemon-config\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778499    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-systemd\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778526    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8428800c-84eb-4d79-a6f2-313e1ae6ad23-tmp\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778550    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-run-systemd\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778573    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-etc-selinux\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778598    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-multus-cni-dir\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778606    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1d27ec4-585a-4bbb-820c-315e02774e2b-ovnkube-script-lib\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778641    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-run-k8s-cni-cncf-io\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778670    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f02fbba4-deae-4855-8e20-88163660aa6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778707    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f02fbba4-deae-4855-8e20-88163660aa6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778789    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d4nc\" (UniqueName: \"kubernetes.io/projected/54626d74-62d9-4fea-ae0a-1a73a8f7fee3-kube-api-access-7d4nc\") pod \"node-resolver-vx8bd\" (UID: \"54626d74-62d9-4fea-ae0a-1a73a8f7fee3\") " pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.778811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778796    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-systemd\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778830    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-run-systemd\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778890    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-sys\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778991    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-systemd-units\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.778995    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-sys\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779020    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779037    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-systemd-units\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779056    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-var-lib-openvswitch\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779094    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-var-lib-openvswitch\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779095    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrsnr\" (UniqueName: \"kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr\") pod \"network-check-target-r8f9c\" (UID: \"0c54c59c-5570-4620-ae15-9fb594f16b2f\") " pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779149    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-cnibin\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779175    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-run-netns\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779206    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54626d74-62d9-4fea-ae0a-1a73a8f7fee3-hosts-file\") pod \"node-resolver-vx8bd\" (UID: \"54626d74-62d9-4fea-ae0a-1a73a8f7fee3\") " pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779254    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-modprobe-d\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779272    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54626d74-62d9-4fea-ae0a-1a73a8f7fee3-hosts-file\") pod \"node-resolver-vx8bd\" (UID: \"54626d74-62d9-4fea-ae0a-1a73a8f7fee3\") " pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779295    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-tuned\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779316    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88de4096-fcfd-4023-9826-b3cc4301b195-host-slash\") pod \"iptables-alerter-94rt8\" (UID: \"88de4096-fcfd-4023-9826-b3cc4301b195\") " pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779335    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-modprobe-d\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.779338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779350    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-registration-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779378    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-device-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779379    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88de4096-fcfd-4023-9826-b3cc4301b195-host-slash\") pod \"iptables-alerter-94rt8\" (UID: \"88de4096-fcfd-4023-9826-b3cc4301b195\") " pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779405    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-lib-modules\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779449    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-os-release\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779472    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcbxb\" (UniqueName: \"kubernetes.io/projected/f02fbba4-deae-4855-8e20-88163660aa6e-kube-api-access-lcbxb\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779488    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-var-lib-kubelet\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779514    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-run-multus-certs\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779533    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-kubernetes\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779544    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-os-release\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779549    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-run\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779584    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1d27ec4-585a-4bbb-820c-315e02774e2b-env-overrides\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779590    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-run\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779640    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-kubernetes\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779645    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-os-release\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779676    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/968fd454-81ec-4346-85a0-967a08e76ee6-cni-binary-copy\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779706    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-etc-openvswitch\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.780440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779732    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-lib-modules\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779749    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee67f1bd-8263-43a0-b0fd-d552419d88a6-konnectivity-ca\") pod \"konnectivity-agent-hvmch\" (UID: \"ee67f1bd-8263-43a0-b0fd-d552419d88a6\") " pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779776    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779802    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88de4096-fcfd-4023-9826-b3cc4301b195-iptables-alerter-script\") pod \"iptables-alerter-94rt8\" (UID: \"88de4096-fcfd-4023-9826-b3cc4301b195\") " pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779830    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779856    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jglpj\" (UniqueName: \"kubernetes.io/projected/f1d27ec4-585a-4bbb-820c-315e02774e2b-kube-api-access-jglpj\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779883    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-dbus\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779912    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-etc-kubernetes\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779940    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-slash\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779972    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-log-socket\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780001    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780008    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1d27ec4-585a-4bbb-820c-315e02774e2b-env-overrides\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.779804    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-etc-openvswitch\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780178    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-cnibin\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780215    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780245    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f02fbba4-deae-4855-8e20-88163660aa6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780272    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-kubelet-config\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:04.781359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780276    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-log-socket\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780312    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-run-netns\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780321    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-slash\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780342    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780368    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-sys-fs\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780392    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fxg5\" (UniqueName: \"kubernetes.io/projected/9b693810-9db1-4af4-a258-64e3e6728835-kube-api-access-2fxg5\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780423    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-run-openvswitch\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780428    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780436    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee67f1bd-8263-43a0-b0fd-d552419d88a6-konnectivity-ca\") pod \"konnectivity-agent-hvmch\" (UID: \"ee67f1bd-8263-43a0-b0fd-d552419d88a6\") " pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780449    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-run-ovn\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780472    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-cnibin\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780477    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-run-ovn-kubernetes\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780503    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88de4096-fcfd-4023-9826-b3cc4301b195-iptables-alerter-script\") pod \"iptables-alerter-94rt8\" (UID: \"88de4096-fcfd-4023-9826-b3cc4301b195\") " pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780509    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xgmd\" (UniqueName: \"kubernetes.io/projected/6d37267a-ce1b-4238-ad69-34ada6577972-kube-api-access-6xgmd\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780524    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-run-ovn\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780540    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780562    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-run-openvswitch\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.782293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780562    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-run-ovn-kubernetes\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780569    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-multus-socket-dir-parent\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780604    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1d27ec4-585a-4bbb-820c-315e02774e2b-ovn-node-metrics-cert\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780609    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-run-netns\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780633    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3c4df1df-58f8-48cf-b5cc-2ca9669d7f32-serviceca\") pod \"node-ca-ljv28\" (UID: \"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32\") " pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780660    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-multus-conf-dir\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780691    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqqhl\" (UniqueName: \"kubernetes.io/projected/968fd454-81ec-4346-85a0-967a08e76ee6-kube-api-access-mqqhl\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54626d74-62d9-4fea-ae0a-1a73a8f7fee3-tmp-dir\") pod \"node-resolver-vx8bd\" (UID: \"54626d74-62d9-4fea-ae0a-1a73a8f7fee3\") " pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780754    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw6nm\" (UniqueName: \"kubernetes.io/projected/8428800c-84eb-4d79-a6f2-313e1ae6ad23-kube-api-access-rw6nm\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780801    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c4df1df-58f8-48cf-b5cc-2ca9669d7f32-host\") pod \"node-ca-ljv28\" (UID: \"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32\") " pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780830    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-system-cni-dir\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780858    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-sysctl-conf\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780887    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-cni-bin\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780912    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-cni-netd\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780937    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee67f1bd-8263-43a0-b0fd-d552419d88a6-agent-certs\") pod \"konnectivity-agent-hvmch\" (UID: \"ee67f1bd-8263-43a0-b0fd-d552419d88a6\") " pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780963    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-host\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780975    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f02fbba4-deae-4855-8e20-88163660aa6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.783184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.780991    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1d27ec4-585a-4bbb-820c-315e02774e2b-ovnkube-config\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.781015    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3c4df1df-58f8-48cf-b5cc-2ca9669d7f32-serviceca\") pod \"node-ca-ljv28\" (UID: \"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32\") " pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.781039    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-cni-netd\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.781041    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f02fbba4-deae-4855-8e20-88163660aa6e-system-cni-dir\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.781120    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1d27ec4-585a-4bbb-820c-315e02774e2b-host-cni-bin\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.781200    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-sysctl-conf\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.781259    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8428800c-84eb-4d79-a6f2-313e1ae6ad23-host\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.781356    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c4df1df-58f8-48cf-b5cc-2ca9669d7f32-host\") pod \"node-ca-ljv28\" (UID: \"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32\") " pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.781513    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54626d74-62d9-4fea-ae0a-1a73a8f7fee3-tmp-dir\") pod \"node-resolver-vx8bd\" (UID: \"54626d74-62d9-4fea-ae0a-1a73a8f7fee3\") " pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.781601    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1d27ec4-585a-4bbb-820c-315e02774e2b-ovnkube-config\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.782968    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8428800c-84eb-4d79-a6f2-313e1ae6ad23-tmp\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.783047    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8428800c-84eb-4d79-a6f2-313e1ae6ad23-etc-tuned\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.783742 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.783671    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1d27ec4-585a-4bbb-820c-315e02774e2b-ovn-node-metrics-cert\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.784332 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.784304    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee67f1bd-8263-43a0-b0fd-d552419d88a6-agent-certs\") pod \"konnectivity-agent-hvmch\" (UID: \"ee67f1bd-8263-43a0-b0fd-d552419d88a6\") " pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:04.796779 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.796736    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjtxn\" (UniqueName: \"kubernetes.io/projected/88de4096-fcfd-4023-9826-b3cc4301b195-kube-api-access-wjtxn\") pod \"iptables-alerter-94rt8\" (UID: \"88de4096-fcfd-4023-9826-b3cc4301b195\") " pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:04.798013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.797984    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw6nm\" (UniqueName: \"kubernetes.io/projected/8428800c-84eb-4d79-a6f2-313e1ae6ad23-kube-api-access-rw6nm\") pod \"tuned-sgdhl\" (UID: \"8428800c-84eb-4d79-a6f2-313e1ae6ad23\") " pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:04.798759 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.798731    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jglpj\" (UniqueName: \"kubernetes.io/projected/f1d27ec4-585a-4bbb-820c-315e02774e2b-kube-api-access-jglpj\") pod \"ovnkube-node-dkxjp\" (UID: \"f1d27ec4-585a-4bbb-820c-315e02774e2b\") " pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:04.799896 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.799573    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb5hs\" (UniqueName: \"kubernetes.io/projected/3c4df1df-58f8-48cf-b5cc-2ca9669d7f32-kube-api-access-gb5hs\") pod \"node-ca-ljv28\" (UID: \"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32\") " pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:04.802063 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.802038    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d4nc\" (UniqueName: \"kubernetes.io/projected/54626d74-62d9-4fea-ae0a-1a73a8f7fee3-kube-api-access-7d4nc\") pod \"node-resolver-vx8bd\" (UID: \"54626d74-62d9-4fea-ae0a-1a73a8f7fee3\") " pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.805068 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.805018    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcbxb\" (UniqueName: \"kubernetes.io/projected/f02fbba4-deae-4855-8e20-88163660aa6e-kube-api-access-lcbxb\") pod \"multus-additional-cni-plugins-kbt4b\" (UID: \"f02fbba4-deae-4855-8e20-88163660aa6e\") " pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:04.822391 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.822323    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal" event={"ID":"47b82ae00a4e61d8c8286ec65e9c9c5f","Type":"ContainerStarted","Data":"dba05995f16ea4daffc24a9ab2c028891cb3ce1bdc120dbeedd7d6ddbf412c8b"}
Jul 27 17:21:04.829867 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.829819    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal" event={"ID":"652ec2025569d089187b9f40ad489f57","Type":"ContainerStarted","Data":"e0b0e34b34df824283df0dc9764a35277a583cebb2628e552b41750b2e5e8264"}
Jul 27 17:21:04.881356 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881306    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xgmd\" (UniqueName: \"kubernetes.io/projected/6d37267a-ce1b-4238-ad69-34ada6577972-kube-api-access-6xgmd\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:04.881356 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881364    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.881614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881398    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-multus-socket-dir-parent\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881431    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-multus-conf-dir\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881461    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqqhl\" (UniqueName: \"kubernetes.io/projected/968fd454-81ec-4346-85a0-967a08e76ee6-kube-api-access-mqqhl\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881486    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.881614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881498    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-var-lib-cni-multus\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881547    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-var-lib-cni-multus\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881578    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-socket-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.881614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881599    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-multus-conf-dir\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881608    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-system-cni-dir\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881645    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-var-lib-cni-bin\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881670    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-hostroot\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881546    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-multus-socket-dir-parent\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881698    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/968fd454-81ec-4346-85a0-967a08e76ee6-multus-daemon-config\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881748    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-etc-selinux\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881756    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-hostroot\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881772    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-multus-cni-dir\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881791    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-run-k8s-cni-cncf-io\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881803    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-var-lib-cni-bin\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881812    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsnr\" (UniqueName: \"kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr\") pod \"network-check-target-r8f9c\" (UID: \"0c54c59c-5570-4620-ae15-9fb594f16b2f\") " pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881831    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-cnibin\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881772    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-socket-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881846    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-run-netns\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881873    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-system-cni-dir\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881880    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-run-netns\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881893    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-registration-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881908    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-run-k8s-cni-cncf-io\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.881977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881931    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-device-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881943    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-cnibin\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881965    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-var-lib-kubelet\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881975    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-registration-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881988    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-etc-selinux\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.881994    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-run-multus-certs\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882028    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-os-release\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882045    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-var-lib-kubelet\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882058    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/968fd454-81ec-4346-85a0-967a08e76ee6-cni-binary-copy\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882078    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-multus-cni-dir\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882090    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882124    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-dbus\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882126    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-os-release\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882152    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-etc-kubernetes\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882186    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-kubelet-config\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.882209    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882216    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:04.882599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882237    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-sys-fs\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.882276    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs podName:6d37267a-ce1b-4238-ad69-34ada6577972 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:05.38224854 +0000 UTC m=+3.090396838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs") pod "network-metrics-daemon-xk28w" (UID: "6d37267a-ce1b-4238-ad69-34ada6577972") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882288    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-sys-fs\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882292    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fxg5\" (UniqueName: \"kubernetes.io/projected/9b693810-9db1-4af4-a258-64e3e6728835-kube-api-access-2fxg5\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882091    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-host-run-multus-certs\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882331    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/968fd454-81ec-4346-85a0-967a08e76ee6-multus-daemon-config\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882339    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b693810-9db1-4af4-a258-64e3e6728835-device-dir\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882353    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-kubelet-config\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.882371    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882406    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-dbus\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.882423    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret podName:5022fa8c-0fd5-4291-ab27-15b6ddae7ad2 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:05.382409634 +0000 UTC m=+3.090557932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret") pod "global-pull-secret-syncer-lc6cl" (UID: "5022fa8c-0fd5-4291-ab27-15b6ddae7ad2") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882448    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/968fd454-81ec-4346-85a0-967a08e76ee6-etc-kubernetes\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.883510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.882791    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/968fd454-81ec-4346-85a0-967a08e76ee6-cni-binary-copy\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.892571 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.892364    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:21:04.892571 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.892395    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:21:04.892571 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.892410    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hrsnr for pod openshift-network-diagnostics/network-check-target-r8f9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:04.892571 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:04.892489    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr podName:0c54c59c-5570-4620-ae15-9fb594f16b2f nodeName:}" failed. No retries permitted until 2026-07-27 17:21:05.392465818 +0000 UTC m=+3.100614122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hrsnr" (UniqueName: "kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr") pod "network-check-target-r8f9c" (UID: "0c54c59c-5570-4620-ae15-9fb594f16b2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:04.894351 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.894311    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xgmd\" (UniqueName: \"kubernetes.io/projected/6d37267a-ce1b-4238-ad69-34ada6577972-kube-api-access-6xgmd\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:04.895563 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.895541    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqqhl\" (UniqueName: \"kubernetes.io/projected/968fd454-81ec-4346-85a0-967a08e76ee6-kube-api-access-mqqhl\") pod \"multus-bfl69\" (UID: \"968fd454-81ec-4346-85a0-967a08e76ee6\") " pod="openshift-multus/multus-bfl69"
Jul 27 17:21:04.895676 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.895575    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fxg5\" (UniqueName: \"kubernetes.io/projected/9b693810-9db1-4af4-a258-64e3e6728835-kube-api-access-2fxg5\") pod \"aws-ebs-csi-driver-node-2fwxg\" (UID: \"9b693810-9db1-4af4-a258-64e3e6728835\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:04.967613 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.967568    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vx8bd"
Jul 27 17:21:04.985982 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:04.985939    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sgdhl"
Jul 27 17:21:05.011372 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.011330    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-94rt8"
Jul 27 17:21:05.017167 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.017136    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:05.028131 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.028093    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:05.036056 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.036029    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ljv28"
Jul 27 17:21:05.047051 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.047023    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kbt4b"
Jul 27 17:21:05.054125 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.054090    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bfl69"
Jul 27 17:21:05.063029 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.063001    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg"
Jul 27 17:21:05.386965 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.386927    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:05.387119 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.386977    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:05.387119 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:05.387111    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:05.387236 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:05.387109    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:05.387236 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:05.387174    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret podName:5022fa8c-0fd5-4291-ab27-15b6ddae7ad2 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:06.387159134 +0000 UTC m=+4.095307439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret") pod "global-pull-secret-syncer-lc6cl" (UID: "5022fa8c-0fd5-4291-ab27-15b6ddae7ad2") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:05.387236 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:05.387194    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs podName:6d37267a-ce1b-4238-ad69-34ada6577972 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:06.387184296 +0000 UTC m=+4.095332597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs") pod "network-metrics-daemon-xk28w" (UID: "6d37267a-ce1b-4238-ad69-34ada6577972") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:05.488169 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.488123    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsnr\" (UniqueName: \"kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr\") pod \"network-check-target-r8f9c\" (UID: \"0c54c59c-5570-4620-ae15-9fb594f16b2f\") " pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:05.488375 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:05.488335    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:21:05.488375 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:05.488357    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:21:05.488375 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:05.488369    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hrsnr for pod openshift-network-diagnostics/network-check-target-r8f9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:05.488501 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:05.488427    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr podName:0c54c59c-5570-4620-ae15-9fb594f16b2f nodeName:}" failed. No retries permitted until 2026-07-27 17:21:06.488411095 +0000 UTC m=+4.196559394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hrsnr" (UniqueName: "kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr") pod "network-check-target-r8f9c" (UID: "0c54c59c-5570-4620-ae15-9fb594f16b2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:05.720847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.720735    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 17:16:05 +0000 UTC" deadline="2028-04-06 10:20:53.374074812 +0000 UTC"
Jul 27 17:21:05.720847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.720792    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14848h59m47.653287137s"
Jul 27 17:21:05.817709 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.817651    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:05.817709 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:05.817690    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:05.817945 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:05.817828    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:05.818003 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:05.817981    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:05.900003 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:05.899967    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4df1df_58f8_48cf_b5cc_2ca9669d7f32.slice/crio-b40d93ae295fe26c6c5f6722eb74c981c849de3f1d4789a3cfb57493d9a5a57e WatchSource:0}: Error finding container b40d93ae295fe26c6c5f6722eb74c981c849de3f1d4789a3cfb57493d9a5a57e: Status 404 returned error can't find the container with id b40d93ae295fe26c6c5f6722eb74c981c849de3f1d4789a3cfb57493d9a5a57e
Jul 27 17:21:05.903536 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:05.903478    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod968fd454_81ec_4346_85a0_967a08e76ee6.slice/crio-f1f4c178088b6149b6517e636195ddf761c627cefd5d2d327a91dbc7c1314ccb WatchSource:0}: Error finding container f1f4c178088b6149b6517e636195ddf761c627cefd5d2d327a91dbc7c1314ccb: Status 404 returned error can't find the container with id f1f4c178088b6149b6517e636195ddf761c627cefd5d2d327a91dbc7c1314ccb
Jul 27 17:21:05.905045 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:05.905020    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee67f1bd_8263_43a0_b0fd_d552419d88a6.slice/crio-91d8d45ce4a901ada84edf93abf7906005cb09ec9bce641d86f964608dfa8ac4 WatchSource:0}: Error finding container 91d8d45ce4a901ada84edf93abf7906005cb09ec9bce641d86f964608dfa8ac4: Status 404 returned error can't find the container with id 91d8d45ce4a901ada84edf93abf7906005cb09ec9bce641d86f964608dfa8ac4
Jul 27 17:21:05.908376 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:05.908251    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02fbba4_deae_4855_8e20_88163660aa6e.slice/crio-fc0b4428dee63bec8c5ee95ff5c5f96ac6982d719dd56697d1918d32e78e7d8f WatchSource:0}: Error finding container fc0b4428dee63bec8c5ee95ff5c5f96ac6982d719dd56697d1918d32e78e7d8f: Status 404 returned error can't find the container with id fc0b4428dee63bec8c5ee95ff5c5f96ac6982d719dd56697d1918d32e78e7d8f
Jul 27 17:21:05.909352 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:05.909327    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b693810_9db1_4af4_a258_64e3e6728835.slice/crio-9b5fa0299e29185f6f54dbd159f5019370063d45a5a01388a9c4951387c3a6c5 WatchSource:0}: Error finding container 9b5fa0299e29185f6f54dbd159f5019370063d45a5a01388a9c4951387c3a6c5: Status 404 returned error can't find the container with id 9b5fa0299e29185f6f54dbd159f5019370063d45a5a01388a9c4951387c3a6c5
Jul 27 17:21:05.910754 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:05.910618    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d27ec4_585a_4bbb_820c_315e02774e2b.slice/crio-93b17c9415fe77b02fd4ee122b10ee04ac63ee86931f09ede22df2c934b1d081 WatchSource:0}: Error finding container 93b17c9415fe77b02fd4ee122b10ee04ac63ee86931f09ede22df2c934b1d081: Status 404 returned error can't find the container with id 93b17c9415fe77b02fd4ee122b10ee04ac63ee86931f09ede22df2c934b1d081
Jul 27 17:21:05.912007 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:05.911883    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88de4096_fcfd_4023_9826_b3cc4301b195.slice/crio-7b90b762cbd72b34dd36004247e2264468e43e851b4d54b94163288cf33d482b WatchSource:0}: Error finding container 7b90b762cbd72b34dd36004247e2264468e43e851b4d54b94163288cf33d482b: Status 404 returned error can't find the container with id 7b90b762cbd72b34dd36004247e2264468e43e851b4d54b94163288cf33d482b
Jul 27 17:21:05.913142 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:05.913002    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8428800c_84eb_4d79_a6f2_313e1ae6ad23.slice/crio-598c3ff77db1d43fc91877c020e49dca0f2b3eb83411f6cd4ee8ba0819526e24 WatchSource:0}: Error finding container 598c3ff77db1d43fc91877c020e49dca0f2b3eb83411f6cd4ee8ba0819526e24: Status 404 returned error can't find the container with id 598c3ff77db1d43fc91877c020e49dca0f2b3eb83411f6cd4ee8ba0819526e24
Jul 27 17:21:06.398327 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.398021    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:06.398327 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.398283    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:06.398327 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:06.398200    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:06.398598 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:06.398373    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:06.398598 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:06.398377    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret podName:5022fa8c-0fd5-4291-ab27-15b6ddae7ad2 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:08.398353669 +0000 UTC m=+6.106501977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret") pod "global-pull-secret-syncer-lc6cl" (UID: "5022fa8c-0fd5-4291-ab27-15b6ddae7ad2") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:06.398598 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:06.398429    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs podName:6d37267a-ce1b-4238-ad69-34ada6577972 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:08.398412233 +0000 UTC m=+6.106560535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs") pod "network-metrics-daemon-xk28w" (UID: "6d37267a-ce1b-4238-ad69-34ada6577972") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:06.499364 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.499318    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsnr\" (UniqueName: \"kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr\") pod \"network-check-target-r8f9c\" (UID: \"0c54c59c-5570-4620-ae15-9fb594f16b2f\") " pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:06.499570 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:06.499528    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:21:06.499570 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:06.499560    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:21:06.499686 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:06.499575    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hrsnr for pod openshift-network-diagnostics/network-check-target-r8f9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:06.499686 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:06.499643    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr podName:0c54c59c-5570-4620-ae15-9fb594f16b2f nodeName:}" failed. No retries permitted until 2026-07-27 17:21:08.499622252 +0000 UTC m=+6.207770564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hrsnr" (UniqueName: "kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr") pod "network-check-target-r8f9c" (UID: "0c54c59c-5570-4620-ae15-9fb594f16b2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:06.722004 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.721876    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 17:16:05 +0000 UTC" deadline="2028-04-26 19:50:49.236173162 +0000 UTC"
Jul 27 17:21:06.722004 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.721948    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15338h29m42.514229436s"
Jul 27 17:21:06.819613 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.819581    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:06.819823 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:06.819715    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:06.842844 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.842802    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal" event={"ID":"47b82ae00a4e61d8c8286ec65e9c9c5f","Type":"ContainerStarted","Data":"6ce01ddfc0dce8a46b725ed49c350f705227b8e4999431c5cffc899c9a492fa2"}
Jul 27 17:21:06.847739 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.847635    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg" event={"ID":"9b693810-9db1-4af4-a258-64e3e6728835","Type":"ContainerStarted","Data":"9b5fa0299e29185f6f54dbd159f5019370063d45a5a01388a9c4951387c3a6c5"}
Jul 27 17:21:06.853780 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.853627    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp" event={"ID":"f1d27ec4-585a-4bbb-820c-315e02774e2b","Type":"ContainerStarted","Data":"93b17c9415fe77b02fd4ee122b10ee04ac63ee86931f09ede22df2c934b1d081"}
Jul 27 17:21:06.858943 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.858908    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-94rt8" event={"ID":"88de4096-fcfd-4023-9826-b3cc4301b195","Type":"ContainerStarted","Data":"7b90b762cbd72b34dd36004247e2264468e43e851b4d54b94163288cf33d482b"}
Jul 27 17:21:06.864468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.864421    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hvmch" event={"ID":"ee67f1bd-8263-43a0-b0fd-d552419d88a6","Type":"ContainerStarted","Data":"91d8d45ce4a901ada84edf93abf7906005cb09ec9bce641d86f964608dfa8ac4"}
Jul 27 17:21:06.870999 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.870919    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bfl69" event={"ID":"968fd454-81ec-4346-85a0-967a08e76ee6","Type":"ContainerStarted","Data":"f1f4c178088b6149b6517e636195ddf761c627cefd5d2d327a91dbc7c1314ccb"}
Jul 27 17:21:06.880778 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.880699    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vx8bd" event={"ID":"54626d74-62d9-4fea-ae0a-1a73a8f7fee3","Type":"ContainerStarted","Data":"25cfcf84b9589bdc18b56906da18fbd5d3e92ee8910d3e2e7a2b5ee2740732ff"}
Jul 27 17:21:06.887137 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.887094    2584 generic.go:356] "Generic (PLEG): container finished" podID="652ec2025569d089187b9f40ad489f57" containerID="7164a7eb1066cb27af796a48964857748acefca6aa1915fb5699db9fb13dfdc9" exitCode=0
Jul 27 17:21:06.887304 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.887203    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal" event={"ID":"652ec2025569d089187b9f40ad489f57","Type":"ContainerDied","Data":"7164a7eb1066cb27af796a48964857748acefca6aa1915fb5699db9fb13dfdc9"}
Jul 27 17:21:06.901886 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.901846    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sgdhl" event={"ID":"8428800c-84eb-4d79-a6f2-313e1ae6ad23","Type":"ContainerStarted","Data":"598c3ff77db1d43fc91877c020e49dca0f2b3eb83411f6cd4ee8ba0819526e24"}
Jul 27 17:21:06.905989 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.905927    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-68.ec2.internal" podStartSLOduration=2.905901955 podStartE2EDuration="2.905901955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:06.861940594 +0000 UTC m=+4.570088917" watchObservedRunningTime="2026-07-27 17:21:06.905901955 +0000 UTC m=+4.614050277"
Jul 27 17:21:06.913951 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.913912    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kbt4b" event={"ID":"f02fbba4-deae-4855-8e20-88163660aa6e","Type":"ContainerStarted","Data":"fc0b4428dee63bec8c5ee95ff5c5f96ac6982d719dd56697d1918d32e78e7d8f"}
Jul 27 17:21:06.917012 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:06.916975    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ljv28" event={"ID":"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32","Type":"ContainerStarted","Data":"b40d93ae295fe26c6c5f6722eb74c981c849de3f1d4789a3cfb57493d9a5a57e"}
Jul 27 17:21:07.816862 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:07.816823    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:07.817396 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:07.816970    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:07.817469 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:07.817396    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:07.817538 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:07.817510    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:07.942277 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:07.942198    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal" event={"ID":"652ec2025569d089187b9f40ad489f57","Type":"ContainerStarted","Data":"0a409ea8a152988ae858602886c498573fcb8cf2ef1561dfbfbc0b122db6ada6"}
Jul 27 17:21:08.423004 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:08.422145    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:08.423004 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:08.422214    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:08.423004 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:08.422368    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:08.423004 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:08.422433    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret podName:5022fa8c-0fd5-4291-ab27-15b6ddae7ad2 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:12.422414354 +0000 UTC m=+10.130562668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret") pod "global-pull-secret-syncer-lc6cl" (UID: "5022fa8c-0fd5-4291-ab27-15b6ddae7ad2") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:08.423004 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:08.422863    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:08.423004 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:08.422920    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs podName:6d37267a-ce1b-4238-ad69-34ada6577972 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:12.422903246 +0000 UTC m=+10.131051548 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs") pod "network-metrics-daemon-xk28w" (UID: "6d37267a-ce1b-4238-ad69-34ada6577972") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:08.523564 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:08.522797    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsnr\" (UniqueName: \"kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr\") pod \"network-check-target-r8f9c\" (UID: \"0c54c59c-5570-4620-ae15-9fb594f16b2f\") " pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:08.523564 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:08.523031    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:21:08.523564 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:08.523052    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:21:08.523564 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:08.523067    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hrsnr for pod openshift-network-diagnostics/network-check-target-r8f9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:08.523564 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:08.523136    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr podName:0c54c59c-5570-4620-ae15-9fb594f16b2f nodeName:}" failed. No retries permitted until 2026-07-27 17:21:12.523114483 +0000 UTC m=+10.231262788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hrsnr" (UniqueName: "kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr") pod "network-check-target-r8f9c" (UID: "0c54c59c-5570-4620-ae15-9fb594f16b2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:08.820040 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:08.819939    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:08.820519 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:08.820147    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:09.818258 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:09.817501    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:09.818258 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:09.817639    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:09.818258 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:09.818090    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:09.818258 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:09.818205    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:10.820542 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:10.819982    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:10.820542 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:10.820108    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:11.817337 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:11.817296    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:11.817540 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:11.817436    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:11.817791 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:11.817770    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:11.818776 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:11.817886    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:12.454504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:12.454462    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:12.454985 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:12.454555    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:12.454985 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:12.454709    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:12.454985 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:12.454772    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs podName:6d37267a-ce1b-4238-ad69-34ada6577972 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:20.45475236 +0000 UTC m=+18.162900659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs") pod "network-metrics-daemon-xk28w" (UID: "6d37267a-ce1b-4238-ad69-34ada6577972") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:12.454985 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:12.454767    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:12.454985 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:12.454842    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret podName:5022fa8c-0fd5-4291-ab27-15b6ddae7ad2 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:20.454822709 +0000 UTC m=+18.162971020 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret") pod "global-pull-secret-syncer-lc6cl" (UID: "5022fa8c-0fd5-4291-ab27-15b6ddae7ad2") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:12.556134 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:12.555507    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsnr\" (UniqueName: \"kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr\") pod \"network-check-target-r8f9c\" (UID: \"0c54c59c-5570-4620-ae15-9fb594f16b2f\") " pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:12.556134 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:12.555664    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:21:12.556134 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:12.555708    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:21:12.556134 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:12.555722    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hrsnr for pod openshift-network-diagnostics/network-check-target-r8f9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:12.556134 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:12.555791    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr podName:0c54c59c-5570-4620-ae15-9fb594f16b2f nodeName:}" failed. No retries permitted until 2026-07-27 17:21:20.555771625 +0000 UTC m=+18.263919924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hrsnr" (UniqueName: "kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr") pod "network-check-target-r8f9c" (UID: "0c54c59c-5570-4620-ae15-9fb594f16b2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:12.818797 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:12.818724    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:12.818919 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:12.818862    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:13.817791 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:13.817747    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:13.818266 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:13.817860    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:13.818266 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:13.818068    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:13.818266 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:13.818188    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:14.820085 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:14.820052    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:14.820572 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:14.820187    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:15.817124 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:15.817076    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:15.817272 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:15.817217    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:15.817610 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:15.817568    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:15.817791 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:15.817660    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:16.817564 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:16.817519    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:16.818042 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:16.817641    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:17.817784 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:17.817741    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:17.818241 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:17.817741    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:17.818241 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:17.817893    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:17.818241 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:17.818011    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:18.817830 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:18.817797    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:18.818254 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:18.817924    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:19.817192 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:19.817151    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:19.817453 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:19.817151    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:19.817453 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:19.817279    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:19.817453 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:19.817395    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:20.519576 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:20.519533    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:20.520197 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:20.519601    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:20.520197 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:20.519730    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:20.520197 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:20.519768    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:20.520197 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:20.519819    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs podName:6d37267a-ce1b-4238-ad69-34ada6577972 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:36.519798448 +0000 UTC m=+34.227946748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs") pod "network-metrics-daemon-xk28w" (UID: "6d37267a-ce1b-4238-ad69-34ada6577972") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:20.520197 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:20.519840    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret podName:5022fa8c-0fd5-4291-ab27-15b6ddae7ad2 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:36.519829573 +0000 UTC m=+34.227977870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret") pod "global-pull-secret-syncer-lc6cl" (UID: "5022fa8c-0fd5-4291-ab27-15b6ddae7ad2") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:20.620663 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:20.620620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsnr\" (UniqueName: \"kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr\") pod \"network-check-target-r8f9c\" (UID: \"0c54c59c-5570-4620-ae15-9fb594f16b2f\") " pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:20.620901 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:20.620826    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:21:20.620901 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:20.620852    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:21:20.620901 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:20.620864    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hrsnr for pod openshift-network-diagnostics/network-check-target-r8f9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:20.621049 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:20.620929    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr podName:0c54c59c-5570-4620-ae15-9fb594f16b2f nodeName:}" failed. No retries permitted until 2026-07-27 17:21:36.620915059 +0000 UTC m=+34.329063357 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hrsnr" (UniqueName: "kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr") pod "network-check-target-r8f9c" (UID: "0c54c59c-5570-4620-ae15-9fb594f16b2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:20.817386 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:20.817347    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:20.817580 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:20.817484    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:21.817510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:21.817473    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:21.818013 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:21.817611    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:21.818013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:21.817789    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:21.818013 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:21.817938    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:22.818428 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:22.818392    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:22.818882 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:22.818525    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:23.817233 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:23.817191    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:23.817414 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:23.817189    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:23.817414 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:23.817310    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:23.817505 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:23.817421    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:24.817706 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:24.817660    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:24.818144 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:24.817793    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:25.817553 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.817225    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:25.817863 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.817294    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:25.817863 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:25.817635    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:25.817863 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:25.817792    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:25.981894 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.981850    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg" event={"ID":"9b693810-9db1-4af4-a258-64e3e6728835","Type":"ContainerStarted","Data":"1afaf6b1558bf0889edf76b0851e1f29046c5404f1affdcdac38e3b1da538e25"}
Jul 27 17:21:25.985657 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.985623    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp" event={"ID":"f1d27ec4-585a-4bbb-820c-315e02774e2b","Type":"ContainerStarted","Data":"e81bb2dcb85c829ef4c1c4b6fe4979117669b466ac0452c2652d37e5b2701140"}
Jul 27 17:21:25.985829 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.985735    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp" event={"ID":"f1d27ec4-585a-4bbb-820c-315e02774e2b","Type":"ContainerStarted","Data":"22634b64e25eaf02b9720d8660eb370a478ce1b5c9d8afa03c4b21a4cbc0b0ac"}
Jul 27 17:21:25.985829 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.985791    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp" event={"ID":"f1d27ec4-585a-4bbb-820c-315e02774e2b","Type":"ContainerStarted","Data":"36bd31957f3fcb5c48752d7a4984e616c48ec1d57b9343cf605e8fb38e1904dd"}
Jul 27 17:21:25.985956 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.985838    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp" event={"ID":"f1d27ec4-585a-4bbb-820c-315e02774e2b","Type":"ContainerStarted","Data":"852062edb341e487ebeed7256d6076ce99dbd9b897c329893fb2dd0240849786"}
Jul 27 17:21:25.985956 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.985886    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp" event={"ID":"f1d27ec4-585a-4bbb-820c-315e02774e2b","Type":"ContainerStarted","Data":"a622cd98139571938aae012d77c12c1a6213ab154b7ba0f1a738544a30867c2b"}
Jul 27 17:21:25.985956 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.985926    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp" event={"ID":"f1d27ec4-585a-4bbb-820c-315e02774e2b","Type":"ContainerStarted","Data":"01e45543c577d879bb8b40a73f3db42233d304ed089c48cbb884d6c276c1a16c"}
Jul 27 17:21:25.987271 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.987208    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hvmch" event={"ID":"ee67f1bd-8263-43a0-b0fd-d552419d88a6","Type":"ContainerStarted","Data":"ef0bb6ebf334ed0c93db03e639565fd7454cb438be332169d34c9c45461ffeb3"}
Jul 27 17:21:25.989330 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.989299    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bfl69" event={"ID":"968fd454-81ec-4346-85a0-967a08e76ee6","Type":"ContainerStarted","Data":"0e433e20bd187783cb75da55e005d2d142cc9a8202ad17323eeddaef47302f06"}
Jul 27 17:21:25.991049 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.991012    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vx8bd" event={"ID":"54626d74-62d9-4fea-ae0a-1a73a8f7fee3","Type":"ContainerStarted","Data":"7e930b640b174375dce7baadee3a5dbed95993a8866d30b56e4ce94fb06ad218"}
Jul 27 17:21:25.993345 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.993316    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sgdhl" event={"ID":"8428800c-84eb-4d79-a6f2-313e1ae6ad23","Type":"ContainerStarted","Data":"00b7c97c0d200664c086521aea75f2c4a486a065a10c946b5872cb63d4143da5"}
Jul 27 17:21:25.995264 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.995240    2584 generic.go:356] "Generic (PLEG): container finished" podID="f02fbba4-deae-4855-8e20-88163660aa6e" containerID="29a62e2f4e984a755a9f9c9e71eb555151ba996fc1ef3fa83a7d76766ab42910" exitCode=0
Jul 27 17:21:25.995378 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.995321    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kbt4b" event={"ID":"f02fbba4-deae-4855-8e20-88163660aa6e","Type":"ContainerDied","Data":"29a62e2f4e984a755a9f9c9e71eb555151ba996fc1ef3fa83a7d76766ab42910"}
Jul 27 17:21:25.997535 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:25.997513    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ljv28" event={"ID":"3c4df1df-58f8-48cf-b5cc-2ca9669d7f32","Type":"ContainerStarted","Data":"8683c0bf8fd36c346d9bf69259db1f1f0849a811d934e78474c9745435b715fa"}
Jul 27 17:21:26.003572 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.003522    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-68.ec2.internal" podStartSLOduration=22.003507192 podStartE2EDuration="22.003507192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:07.959802366 +0000 UTC m=+5.667950686" watchObservedRunningTime="2026-07-27 17:21:26.003507192 +0000 UTC m=+23.711655509"
Jul 27 17:21:26.029380 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.029322    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hvmch" podStartSLOduration=7.816299991 podStartE2EDuration="23.029305803s" totalImagesPullingTime="15.213005812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:26.003645786 +0000 UTC m=+23.711794104" watchObservedRunningTime="2026-07-27 17:21:26.029305803 +0000 UTC m=+23.737454189"
Jul 27 17:21:26.048336 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.048286    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bfl69" podStartSLOduration=4.044989906 podStartE2EDuration="23.048272108s" totalImagesPullingTime="19.003282202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:26.047956202 +0000 UTC m=+23.756104524" watchObservedRunningTime="2026-07-27 17:21:26.048272108 +0000 UTC m=+23.756420427"
Jul 27 17:21:26.064538 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.064491    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vx8bd" podStartSLOduration=4.117293354 podStartE2EDuration="23.064476357s" totalImagesPullingTime="18.947183003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:26.064056682 +0000 UTC m=+23.772205001" watchObservedRunningTime="2026-07-27 17:21:26.064476357 +0000 UTC m=+23.772624675"
Jul 27 17:21:26.104481 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.104420    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sgdhl" podStartSLOduration=4.14605787 podStartE2EDuration="23.104401286s" totalImagesPullingTime="18.958343416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:26.103853748 +0000 UTC m=+23.812002067" watchObservedRunningTime="2026-07-27 17:21:26.104401286 +0000 UTC m=+23.812549606"
Jul 27 17:21:26.126324 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.126225    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ljv28" podStartSLOduration=4.174577057 podStartE2EDuration="23.126206307s" totalImagesPullingTime="18.95162925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:26.126035778 +0000 UTC m=+23.834184113" watchObservedRunningTime="2026-07-27 17:21:26.126206307 +0000 UTC m=+23.834354659"
Jul 27 17:21:26.238984 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.238949    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:26.313818 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.313790    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:26.412940 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.412912    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 17:21:26.726815 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.726648    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T17:21:26.412934501Z","UUID":"4eb513f7-71b6-4648-be08-28fd036022bc","Handler":null,"Name":"","Endpoint":""}
Jul 27 17:21:26.728534 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.728510    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 17:21:26.728656 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.728557    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 17:21:26.817789 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:26.817759    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:26.818069 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:26.817882    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:27.002564 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:27.002454    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg" event={"ID":"9b693810-9db1-4af4-a258-64e3e6728835","Type":"ContainerStarted","Data":"3a456be32966f19567d7315b64d1887d36875cc461a0d4b250b42d37eab03230"}
Jul 27 17:21:27.005630 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:27.005569    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-94rt8" event={"ID":"88de4096-fcfd-4023-9826-b3cc4301b195","Type":"ContainerStarted","Data":"0b0fbfecf09cc3bc020aaa21dca0fa5f11193b1f749046ab43be45465496450c"}
Jul 27 17:21:27.021720 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:27.020950    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-94rt8" podStartSLOduration=5.0804075300000004 podStartE2EDuration="24.020930697s" totalImagesPullingTime="18.940523167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:27.020775083 +0000 UTC m=+24.728923415" watchObservedRunningTime="2026-07-27 17:21:27.020930697 +0000 UTC m=+24.729079017"
Jul 27 17:21:27.146086 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:27.146045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:27.613282 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:27.613256    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:27.817474 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:27.817442    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:27.817752 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:27.817555    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:27.817752 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:27.817608    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:27.817930 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:27.817765    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:27.865996 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:27.865963    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:27.867299 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:27.867278    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:28.008945 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:28.008896    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg" event={"ID":"9b693810-9db1-4af4-a258-64e3e6728835","Type":"ContainerStarted","Data":"361e7ae0c5d9530f64f09e5f48a1d30c92fcab8dc94e8b0a8ed6162a771b15dd"}
Jul 27 17:21:28.012830 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:28.012796    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp" event={"ID":"f1d27ec4-585a-4bbb-820c-315e02774e2b","Type":"ContainerStarted","Data":"8660707b4cc80a0e5bdf84f5c6682794feb74b9b1594b0752f12c1875ce3ed89"}
Jul 27 17:21:28.013502 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:28.013196    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:28.013656 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:28.013637    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hvmch"
Jul 27 17:21:28.032374 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:28.032317    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2fwxg" podStartSLOduration=3.719821954 podStartE2EDuration="25.032296942s" totalImagesPullingTime="21.312474988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:28.026292738 +0000 UTC m=+25.734441055" watchObservedRunningTime="2026-07-27 17:21:28.032296942 +0000 UTC m=+25.740445261"
Jul 27 17:21:28.093895 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:28.093814    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:28.817221 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:28.817188    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:28.817427 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:28.817326    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:29.713108 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:29.712883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:29.781224 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:29.781183    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:29.817254 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:29.817219    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:29.817430 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:29.817220    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:29.817430 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:29.817348    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:29.817519 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:29.817453    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:29.965677 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:29.965595    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:30.098187 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:30.098145    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:30.653206 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:30.653169    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:30.752910 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:30.752815    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:30.817102 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:30.817066    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:30.817235 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:30.817186    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:30.937248 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:30.937208    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:31.022005 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.021913    2584 generic.go:356] "Generic (PLEG): container finished" podID="f02fbba4-deae-4855-8e20-88163660aa6e" containerID="43f405cbff5d82e983e04ed7c8c7ee3d55a4fcf607e6820527cc29d7834203fe" exitCode=0
Jul 27 17:21:31.022005 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.021994    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kbt4b" event={"ID":"f02fbba4-deae-4855-8e20-88163660aa6e","Type":"ContainerDied","Data":"43f405cbff5d82e983e04ed7c8c7ee3d55a4fcf607e6820527cc29d7834203fe"}
Jul 27 17:21:31.026087 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.026060    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp" event={"ID":"f1d27ec4-585a-4bbb-820c-315e02774e2b","Type":"ContainerStarted","Data":"e75ca7d441cf6a5c58682af180e47c5a20f2ac8c752110ba93eafb1864beb0ae"}
Jul 27 17:21:31.026333 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.026321    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:31.037572 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.037518    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:31.043292 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.043270    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:31.074748 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.074668    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp" podStartSLOduration=9.064690342 podStartE2EDuration="28.07464791s" totalImagesPullingTime="19.009957568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:31.072809708 +0000 UTC m=+28.780958026" watchObservedRunningTime="2026-07-27 17:21:31.07464791 +0000 UTC m=+28.782796227"
Jul 27 17:21:31.330226 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.330200    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vx8bd_54626d74-62d9-4fea-ae0a-1a73a8f7fee3/dns-node-resolver/0.log"
Jul 27 17:21:31.345958 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.345930    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:31.411630 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.411589    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:31.686471 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.686212    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:31.778857 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.778806    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:31.817164 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.817131    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:31.817360 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.817144    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:31.817360 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:31.817270    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:31.817476 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:31.817357    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:31.880161 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:31.880133    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:32.029408 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.029306    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:32.029408 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.029396    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:32.048981 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.048946    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:21:32.132160 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.132133    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ljv28_3c4df1df-58f8-48cf-b5cc-2ca9669d7f32/node-ca/0.log"
Jul 27 17:21:32.562638 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.562602    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:32.650897 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.650865    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r8f9c"]
Jul 27 17:21:32.651066 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.650968    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:32.651066 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:32.651046    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:32.654584 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.654546    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lc6cl"]
Jul 27 17:21:32.654747 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.654678    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:32.654822 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:32.654802    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:32.655137 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.655114    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:32.655285 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.655265    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xk28w"]
Jul 27 17:21:32.655358 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:32.655350    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:32.655478 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:32.655444    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:33.033239 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:33.033158    2584 generic.go:356] "Generic (PLEG): container finished" podID="f02fbba4-deae-4855-8e20-88163660aa6e" containerID="8f56fd93069d8023cc37f2a48a590cff03131164f21b92aa62e424668329b900" exitCode=0
Jul 27 17:21:33.033635 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:33.033243    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kbt4b" event={"ID":"f02fbba4-deae-4855-8e20-88163660aa6e","Type":"ContainerDied","Data":"8f56fd93069d8023cc37f2a48a590cff03131164f21b92aa62e424668329b900"}
Jul 27 17:21:33.265827 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:33.265788    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:33.328394 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:33.328364    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:33.817671 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:33.817649    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:33.817826 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:33.817778    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:34.135131 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:34.135099    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:34.213471 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:34.213286    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:34.817306 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:34.817270    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:34.817486 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:34.817270    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:34.817486 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:34.817416    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:34.817602 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:34.817491    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:35.040304 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:35.040266    2584 generic.go:356] "Generic (PLEG): container finished" podID="f02fbba4-deae-4855-8e20-88163660aa6e" containerID="067615af87f61cd84e672b391770f06d8ff4df8e7799db16e4cff584453566f9" exitCode=0
Jul 27 17:21:35.040466 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:35.040321    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kbt4b" event={"ID":"f02fbba4-deae-4855-8e20-88163660aa6e","Type":"ContainerDied","Data":"067615af87f61cd84e672b391770f06d8ff4df8e7799db16e4cff584453566f9"}
Jul 27 17:21:35.816832 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:35.816791    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:35.817288 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:35.816927    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:36.487750 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:36.487717    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:36.538954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:36.538919    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:36.539111 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:36.538961    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:36.539111 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:36.539078    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:36.539111 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:36.539078    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:36.539289 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:36.539146    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret podName:5022fa8c-0fd5-4291-ab27-15b6ddae7ad2 nodeName:}" failed. No retries permitted until 2026-07-27 17:22:08.539129377 +0000 UTC m=+66.247277677 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret") pod "global-pull-secret-syncer-lc6cl" (UID: "5022fa8c-0fd5-4291-ab27-15b6ddae7ad2") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:36.539289 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:36.539162    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs podName:6d37267a-ce1b-4238-ad69-34ada6577972 nodeName:}" failed. No retries permitted until 2026-07-27 17:22:08.539155181 +0000 UTC m=+66.247303478 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs") pod "network-metrics-daemon-xk28w" (UID: "6d37267a-ce1b-4238-ad69-34ada6577972") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:36.587146 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:36.587060    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:36.639895 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:36.639847    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsnr\" (UniqueName: \"kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr\") pod \"network-check-target-r8f9c\" (UID: \"0c54c59c-5570-4620-ae15-9fb594f16b2f\") " pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:36.640089 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:36.640051    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:21:36.640089 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:36.640076    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:21:36.640216 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:36.640097    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hrsnr for pod openshift-network-diagnostics/network-check-target-r8f9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:36.640216 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:36.640170    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr podName:0c54c59c-5570-4620-ae15-9fb594f16b2f nodeName:}" failed. No retries permitted until 2026-07-27 17:22:08.640143101 +0000 UTC m=+66.348291412 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hrsnr" (UniqueName: "kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr") pod "network-check-target-r8f9c" (UID: "0c54c59c-5570-4620-ae15-9fb594f16b2f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:36.817608 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:36.817566    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:36.817608 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:36.817605    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:36.818152 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:36.817715    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:36.818152 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:36.817833    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:37.817070 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:37.817029    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:37.817269 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:37.817164    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:38.073572 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:38.073481    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:38.173811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:38.173778    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:38.305460 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:38.305424    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:38.387961 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:38.387879    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:38.817272 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:38.817239    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:38.817474 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:38.817240    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:38.817474 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:38.817386    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:38.817474 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:38.817449    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:38.977425 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:38.977396    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:39.817187 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:39.817154    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:39.817618 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:39.817257    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:39.818191 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:39.818153    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:39.971102 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:39.971044    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:40.054702 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:40.054652    2584 generic.go:356] "Generic (PLEG): container finished" podID="f02fbba4-deae-4855-8e20-88163660aa6e" containerID="ce2e94a3d2ed55213f10f161db3f47da196126e4e2b3a920de729e436a5e6cac" exitCode=0
Jul 27 17:21:40.054843 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:40.054724    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kbt4b" event={"ID":"f02fbba4-deae-4855-8e20-88163660aa6e","Type":"ContainerDied","Data":"ce2e94a3d2ed55213f10f161db3f47da196126e4e2b3a920de729e436a5e6cac"}
Jul 27 17:21:40.367491 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:40.367388    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:40.817473 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:40.817442    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:40.817993 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:40.817442    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:40.817993 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:40.817601    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:40.817993 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:40.817752    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:41.026025 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:41.025991    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:41.060541 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:41.060503    2584 generic.go:356] "Generic (PLEG): container finished" podID="f02fbba4-deae-4855-8e20-88163660aa6e" containerID="5c00c9c0da237d680fe9f83306957bd631096786368cad046d78944b9b25b0c3" exitCode=0
Jul 27 17:21:41.060541 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:41.060543    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kbt4b" event={"ID":"f02fbba4-deae-4855-8e20-88163660aa6e","Type":"ContainerDied","Data":"5c00c9c0da237d680fe9f83306957bd631096786368cad046d78944b9b25b0c3"}
Jul 27 17:21:41.250299 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:41.250269    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:41.319320 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:41.319286    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:41.569385 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:41.569355    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:41.817746 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:41.817719    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:41.818316 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:41.817838    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:42.066935 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:42.066901    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kbt4b" event={"ID":"f02fbba4-deae-4855-8e20-88163660aa6e","Type":"ContainerStarted","Data":"8830675ab7a9350ea05ab87893b7431a16a21e98864e0c9153d83145e7374cf8"}
Jul 27 17:21:42.098420 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:42.098360    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kbt4b" podStartSLOduration=9.475563107 podStartE2EDuration="39.098342301s" totalImagesPullingTime="29.622779194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:42.097826072 +0000 UTC m=+39.805974390" watchObservedRunningTime="2026-07-27 17:21:42.098342301 +0000 UTC m=+39.806490622"
Jul 27 17:21:42.188802 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:42.188770    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:42.306410 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:42.306383    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:42.776023 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:42.775937    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:42.818149 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:42.818114    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:42.818543 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:42.818222    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:42.818543 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:42.818273    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:42.818543 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:42.818363    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:42.873722 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:42.873663    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:43.440335 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:43.440299    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:43.567269 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:43.567236    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:43.690215 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:43.690172    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:43.789386 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:43.789299    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:43.817384 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:43.817341    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:43.817517 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:43.817460    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:43.864018 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:43.863987    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:44.201504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:44.201471    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:44.426975 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:44.426946    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:44.770722 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:44.770674    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:44.817809 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:44.817774    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:44.817927 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:44.817890    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:44.817978 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:44.817956    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:44.818076 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:44.818059    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:45.172450 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:45.172414    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:45.817796 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:45.817758    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:45.817990 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:45.817865    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:45.943921 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:45.943885    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:46.018405 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:46.018375    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:46.370502 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:46.370468    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:46.768058 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:46.767967    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:46.817664 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:46.817633    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:46.817870 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:46.817800    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:46.817870 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:46.817839    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:46.817994 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:46.817942    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:47.167735 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:47.167674    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:47.575098 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:47.575070    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:47.817157 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:47.817120    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:47.817329 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:47.817225    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:47.965913 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:47.965839    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:48.167677 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:48.167638    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:48.817828 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:48.817791    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:48.817828 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:48.817825    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:48.818265 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:48.817925    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:48.818265 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:48.818040    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:49.385348 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:49.385317    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:49.444585 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:49.444548    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:49.817341 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:49.817311    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:49.817527 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:49.817432    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:50.817568 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:50.817525    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:50.818074 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:50.817525    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:50.818074 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:50.817638    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:50.818074 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:50.817778    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:51.817800 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:51.817764    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:51.818198 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:51.817868    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:52.430839 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:52.430804    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:52.531276 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:52.531244    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:52.590915 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:52.590881    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:52.702419 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:52.702339    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:52.818368 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:52.818330    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:52.818783 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:52.818465    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:52.818783 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:52.818512    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:52.818783 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:52.818602    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:53.095604 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:53.095574    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:53.703867 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:53.703826    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:53.804098 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:53.804039    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:53.817193 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:53.817155    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:53.817357 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:53.817269    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:53.904283 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:53.904248    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:54.117025 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:54.116989    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:54.436954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:54.436873    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:54.591762 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:54.591729    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:54.817802 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:54.817773    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:54.817957 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:54.817779    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:54.817957 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:54.817919    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lc6cl" podUID="5022fa8c-0fd5-4291-ab27-15b6ddae7ad2"
Jul 27 17:21:54.818105 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:54.817970    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xk28w" podUID="6d37267a-ce1b-4238-ad69-34ada6577972"
Jul 27 17:21:55.817763 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:55.817723    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:55.818160 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:21:55.817845    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8f9c" podUID="0c54c59c-5570-4620-ae15-9fb594f16b2f"
Jul 27 17:21:56.166286 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.166184    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:56.314570 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.314536    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:56.418826 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.418751    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:56.517067 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.517032    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:56.581550 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.581527    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-68.ec2.internal" event="NodeReady"
Jul 27 17:21:56.581745 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.581653    2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 17:21:56.722488 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.722404    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2hlk2"]
Jul 27 17:21:56.724269 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.724253    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.727417 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.727395    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4vhhn"]
Jul 27 17:21:56.728917 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.728896    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2k848\""
Jul 27 17:21:56.729056 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.729036    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 17:21:56.729222 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.729207    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 17:21:56.729270 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.729219    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4vhhn"
Jul 27 17:21:56.735450 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.735432    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 17:21:56.735558 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.735435    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 17:21:56.735607 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.735555    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bgcdf\""
Jul 27 17:21:56.735607 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.735586    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 17:21:56.740617 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.740594    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2hlk2"]
Jul 27 17:21:56.743996 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.743972    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4vhhn"]
Jul 27 17:21:56.751930 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.751904    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nrpkt"]
Jul 27 17:21:56.754143 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.754127    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.757125 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.757106    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 17:21:56.757235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.757218    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 17:21:56.757456 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.757430    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 17:21:56.757512 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.757478    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 17:21:56.758652 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.758636    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bfsxf\""
Jul 27 17:21:56.770596 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.770569    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nrpkt"]
Jul 27 17:21:56.817811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.817779    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:21:56.818275 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.817789    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:21:56.820926 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.820903    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfvk\""
Jul 27 17:21:56.821084 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.820900    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 17:21:56.821084 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.820904    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 17:21:56.875474 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.875441    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:56.882575 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882541    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccfb319-8736-425b-bc66-486e3c249e88-data-volume\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.882739 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882584    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccfb319-8736-425b-bc66-486e3c249e88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.882739 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882613    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfp6c\" (UniqueName: \"kubernetes.io/projected/e29ddd7d-78f5-4b66-ad2d-3e5010129212-kube-api-access-nfp6c\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.882739 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882638    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29ddd7d-78f5-4b66-ad2d-3e5010129212-tmp-dir\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.882739 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882671    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ccfb319-8736-425b-bc66-486e3c249e88-crio-socket\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.882739 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882730    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e29ddd7d-78f5-4b66-ad2d-3e5010129212-metrics-tls\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.882954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882774    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e29ddd7d-78f5-4b66-ad2d-3e5010129212-config-volume\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.882954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882798    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cwb8\" (UniqueName: \"kubernetes.io/projected/3ccfb319-8736-425b-bc66-486e3c249e88-kube-api-access-2cwb8\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.882954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882868    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d34c6ca9-5a01-465e-bfba-2b9b8d468e94-cert\") pod \"ingress-canary-4vhhn\" (UID: \"d34c6ca9-5a01-465e-bfba-2b9b8d468e94\") " pod="openshift-ingress-canary/ingress-canary-4vhhn"
Jul 27 17:21:56.882954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882907    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccfb319-8736-425b-bc66-486e3c249e88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.882954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.882928    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b92w\" (UniqueName: \"kubernetes.io/projected/d34c6ca9-5a01-465e-bfba-2b9b8d468e94-kube-api-access-8b92w\") pod \"ingress-canary-4vhhn\" (UID: \"d34c6ca9-5a01-465e-bfba-2b9b8d468e94\") " pod="openshift-ingress-canary/ingress-canary-4vhhn"
Jul 27 17:21:56.949051 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.949011    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75895d884b-kprp5"]
Jul 27 17:21:56.950905 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.950882    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:56.955359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.955331    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hrqfm\""
Jul 27 17:21:56.955519 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.955395    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 27 17:21:56.955642 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.955626    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 27 17:21:56.955732 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.955699    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 27 17:21:56.955732 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.955723    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 27 17:21:56.955949 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.955933    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 27 17:21:56.956135 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.956121    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 27 17:21:56.956280 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.956256    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 27 17:21:56.972146 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.972118    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75895d884b-kprp5"]
Jul 27 17:21:56.983264 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983192    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccfb319-8736-425b-bc66-486e3c249e88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.983264 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983224    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b92w\" (UniqueName: \"kubernetes.io/projected/d34c6ca9-5a01-465e-bfba-2b9b8d468e94-kube-api-access-8b92w\") pod \"ingress-canary-4vhhn\" (UID: \"d34c6ca9-5a01-465e-bfba-2b9b8d468e94\") " pod="openshift-ingress-canary/ingress-canary-4vhhn"
Jul 27 17:21:56.983264 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983259    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccfb319-8736-425b-bc66-486e3c249e88-data-volume\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.983499 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983285    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccfb319-8736-425b-bc66-486e3c249e88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.983499 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983305    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfp6c\" (UniqueName: \"kubernetes.io/projected/e29ddd7d-78f5-4b66-ad2d-3e5010129212-kube-api-access-nfp6c\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.983499 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983322    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29ddd7d-78f5-4b66-ad2d-3e5010129212-tmp-dir\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.983499 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983345    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ccfb319-8736-425b-bc66-486e3c249e88-crio-socket\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.983499 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983372    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e29ddd7d-78f5-4b66-ad2d-3e5010129212-metrics-tls\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.983499 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983474    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e29ddd7d-78f5-4b66-ad2d-3e5010129212-config-volume\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.983825 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983518    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cwb8\" (UniqueName: \"kubernetes.io/projected/3ccfb319-8736-425b-bc66-486e3c249e88-kube-api-access-2cwb8\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.983825 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983560    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d34c6ca9-5a01-465e-bfba-2b9b8d468e94-cert\") pod \"ingress-canary-4vhhn\" (UID: \"d34c6ca9-5a01-465e-bfba-2b9b8d468e94\") " pod="openshift-ingress-canary/ingress-canary-4vhhn"
Jul 27 17:21:56.983825 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983811    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29ddd7d-78f5-4b66-ad2d-3e5010129212-tmp-dir\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.983973 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.983953    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccfb319-8736-425b-bc66-486e3c249e88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.984134 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.984105    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ccfb319-8736-425b-bc66-486e3c249e88-crio-socket\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.984485 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.984463    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccfb319-8736-425b-bc66-486e3c249e88-data-volume\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.984943 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.984919    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e29ddd7d-78f5-4b66-ad2d-3e5010129212-config-volume\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.987637 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.987616    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccfb319-8736-425b-bc66-486e3c249e88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:56.987637 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.987625    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e29ddd7d-78f5-4b66-ad2d-3e5010129212-metrics-tls\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.987807 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.987719    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d34c6ca9-5a01-465e-bfba-2b9b8d468e94-cert\") pod \"ingress-canary-4vhhn\" (UID: \"d34c6ca9-5a01-465e-bfba-2b9b8d468e94\") " pod="openshift-ingress-canary/ingress-canary-4vhhn"
Jul 27 17:21:56.991998 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.991967    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfp6c\" (UniqueName: \"kubernetes.io/projected/e29ddd7d-78f5-4b66-ad2d-3e5010129212-kube-api-access-nfp6c\") pod \"dns-default-2hlk2\" (UID: \"e29ddd7d-78f5-4b66-ad2d-3e5010129212\") " pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:56.993093 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.993073    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b92w\" (UniqueName: \"kubernetes.io/projected/d34c6ca9-5a01-465e-bfba-2b9b8d468e94-kube-api-access-8b92w\") pod \"ingress-canary-4vhhn\" (UID: \"d34c6ca9-5a01-465e-bfba-2b9b8d468e94\") " pod="openshift-ingress-canary/ingress-canary-4vhhn"
Jul 27 17:21:56.993208 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:56.993078    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cwb8\" (UniqueName: \"kubernetes.io/projected/3ccfb319-8736-425b-bc66-486e3c249e88-kube-api-access-2cwb8\") pod \"insights-runtime-extractor-nrpkt\" (UID: \"3ccfb319-8736-425b-bc66-486e3c249e88\") " pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:57.035135 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.035098    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:21:57.040063 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.040017    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4vhhn"
Jul 27 17:21:57.063176 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.063137    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nrpkt"
Jul 27 17:21:57.084618 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.084576    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7w9j\" (UniqueName: \"kubernetes.io/projected/ac688a9e-fd81-48da-bd08-28aa4775478c-kube-api-access-z7w9j\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.084885 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.084655    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-oauth-config\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.084885 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.084674    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-console-config\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.084885 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.084727    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-serving-cert\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.084885 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.084749    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-service-ca\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.084885 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.084796    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-oauth-serving-cert\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.186168 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.186132    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-oauth-config\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.186168 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.186168    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-console-config\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.186422 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.186208    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-serving-cert\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.186422 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.186321    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-service-ca\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.186422 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.186343    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-oauth-serving-cert\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.186422 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.186362    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7w9j\" (UniqueName: \"kubernetes.io/projected/ac688a9e-fd81-48da-bd08-28aa4775478c-kube-api-access-z7w9j\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.187106 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.187036    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-service-ca\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.187106 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.187062    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-oauth-serving-cert\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.187106 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.187031    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-console-config\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.189162 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.189137    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-serving-cert\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.189396 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.189377    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-oauth-config\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.196919 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.196892    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7w9j\" (UniqueName: \"kubernetes.io/projected/ac688a9e-fd81-48da-bd08-28aa4775478c-kube-api-access-z7w9j\") pod \"console-75895d884b-kprp5\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") " pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.231196 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.231160    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nrpkt"]
Jul 27 17:21:57.234287 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:57.234229    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ccfb319_8736_425b_bc66_486e3c249e88.slice/crio-9835732e502588e180377ba62186306d231654d96e44c5b418c9eb38c19ed283 WatchSource:0}: Error finding container 9835732e502588e180377ba62186306d231654d96e44c5b418c9eb38c19ed283: Status 404 returned error can't find the container with id 9835732e502588e180377ba62186306d231654d96e44c5b418c9eb38c19ed283
Jul 27 17:21:57.260315 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.260285    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:21:57.423450 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.423407    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4vhhn"]
Jul 27 17:21:57.428057 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:57.428027    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34c6ca9_5a01_465e_bfba_2b9b8d468e94.slice/crio-424624e64de32790d40a41c58a27d3ff10022952401f8b5e1ed9927faa9a3b93 WatchSource:0}: Error finding container 424624e64de32790d40a41c58a27d3ff10022952401f8b5e1ed9927faa9a3b93: Status 404 returned error can't find the container with id 424624e64de32790d40a41c58a27d3ff10022952401f8b5e1ed9927faa9a3b93
Jul 27 17:21:57.431771 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.431735    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75895d884b-kprp5"]
Jul 27 17:21:57.435069 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:57.435040    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac688a9e_fd81_48da_bd08_28aa4775478c.slice/crio-fc3a7f6e40023eba22e87a80dd9e6cd40014e9a03af8ec3b4655a31045c9b880 WatchSource:0}: Error finding container fc3a7f6e40023eba22e87a80dd9e6cd40014e9a03af8ec3b4655a31045c9b880: Status 404 returned error can't find the container with id fc3a7f6e40023eba22e87a80dd9e6cd40014e9a03af8ec3b4655a31045c9b880
Jul 27 17:21:57.435602 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.435556    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2hlk2"]
Jul 27 17:21:57.442664 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:21:57.442642    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29ddd7d_78f5_4b66_ad2d_3e5010129212.slice/crio-69db9e80eec8137167c2c6b5f3b20530673537b9a686a5ef7b9057f496e1c261 WatchSource:0}: Error finding container 69db9e80eec8137167c2c6b5f3b20530673537b9a686a5ef7b9057f496e1c261: Status 404 returned error can't find the container with id 69db9e80eec8137167c2c6b5f3b20530673537b9a686a5ef7b9057f496e1c261
Jul 27 17:21:57.518413 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.518323    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:57.622321 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.622288    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:57.817551 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.817521    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:21:57.821270 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.821245    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 17:21:57.822561 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.822538    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dgczs\""
Jul 27 17:21:57.822718 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:57.822597    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 17:21:58.107493 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:58.107355    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2hlk2" event={"ID":"e29ddd7d-78f5-4b66-ad2d-3e5010129212","Type":"ContainerStarted","Data":"69db9e80eec8137167c2c6b5f3b20530673537b9a686a5ef7b9057f496e1c261"}
Jul 27 17:21:58.113070 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:58.113026    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75895d884b-kprp5" event={"ID":"ac688a9e-fd81-48da-bd08-28aa4775478c","Type":"ContainerStarted","Data":"fc3a7f6e40023eba22e87a80dd9e6cd40014e9a03af8ec3b4655a31045c9b880"}
Jul 27 17:21:58.115729 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:58.115610    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nrpkt" event={"ID":"3ccfb319-8736-425b-bc66-486e3c249e88","Type":"ContainerStarted","Data":"9835732e502588e180377ba62186306d231654d96e44c5b418c9eb38c19ed283"}
Jul 27 17:21:58.117420 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:21:58.117342    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4vhhn" event={"ID":"d34c6ca9-5a01-465e-bfba-2b9b8d468e94","Type":"ContainerStarted","Data":"424624e64de32790d40a41c58a27d3ff10022952401f8b5e1ed9927faa9a3b93"}
Jul 27 17:22:00.071218 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.071179    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:00.182235 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.182188    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:00.283398 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.283340    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:00.288657 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.288629    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2"]
Jul 27 17:22:00.306034 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.306001    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2"]
Jul 27 17:22:00.306223 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.306159    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2"
Jul 27 17:22:00.309337 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.309304    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 27 17:22:00.309542 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.309393    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cn7wg\""
Jul 27 17:22:00.414958 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.414864    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0968b2c9-caca-4e07-ae42-5422e344e407-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qh2r2\" (UID: \"0968b2c9-caca-4e07-ae42-5422e344e407\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2"
Jul 27 17:22:00.480708 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.480649    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:00.516016 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.515971    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0968b2c9-caca-4e07-ae42-5422e344e407-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qh2r2\" (UID: \"0968b2c9-caca-4e07-ae42-5422e344e407\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2"
Jul 27 17:22:00.519165 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.519139    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0968b2c9-caca-4e07-ae42-5422e344e407-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qh2r2\" (UID: \"0968b2c9-caca-4e07-ae42-5422e344e407\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2"
Jul 27 17:22:00.620750 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.620470    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2"
Jul 27 17:22:00.851366 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.851332    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:00.953489 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:00.953453    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:01.051561 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.051528    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:01.122488 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.122396    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:01.535143 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.535055    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79978b54d7-gkcbp"]
Jul 27 17:22:01.546960 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.546923    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:01.564405 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.564373    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79978b54d7-gkcbp"]
Jul 27 17:22:01.564595 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.564445    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.576533 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.576506    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 27 17:22:01.624502 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.624459    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:01.725950 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.725911    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-config\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.726126 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.725958    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-trusted-ca-bundle\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.726126 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.726034    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nstkb\" (UniqueName: \"kubernetes.io/projected/ca6c17c3-2130-45c6-b429-43aac7e44c5a-kube-api-access-nstkb\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.726126 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.726065    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-service-ca\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.726126 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.726090    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-oauth-serving-cert\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.726298 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.726131    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-serving-cert\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.726298 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.726208    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-oauth-config\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.827017 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.826981    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-oauth-config\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.827198 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.827032    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-config\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.827198 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.827057    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-trusted-ca-bundle\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.827349 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.827243    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nstkb\" (UniqueName: \"kubernetes.io/projected/ca6c17c3-2130-45c6-b429-43aac7e44c5a-kube-api-access-nstkb\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.827349 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.827287    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-service-ca\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.827349 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.827316    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-oauth-serving-cert\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.827497 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.827360    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-serving-cert\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.827915 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.827860    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-config\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.828149 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.828125    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-trusted-ca-bundle\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.828274 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.828089    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-service-ca\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.828664 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.828632    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-oauth-serving-cert\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.830026 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.830001    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-oauth-config\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.830200 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.830179    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-serving-cert\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.838126 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.838094    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nstkb\" (UniqueName: \"kubernetes.io/projected/ca6c17c3-2130-45c6-b429-43aac7e44c5a-kube-api-access-nstkb\") pod \"console-79978b54d7-gkcbp\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") " pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:01.875864 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:01.875822    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:04.050046 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:04.050013    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dkxjp"
Jul 27 17:22:04.180147 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:04.180105    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:04.268106 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:04.268057    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:04.337234 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:04.337194    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:05.428262 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:05.428226    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2"]
Jul 27 17:22:05.448721 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:05.448623    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79978b54d7-gkcbp"]
Jul 27 17:22:06.144663 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.144593    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75895d884b-kprp5" event={"ID":"ac688a9e-fd81-48da-bd08-28aa4775478c","Type":"ContainerStarted","Data":"c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c"}
Jul 27 17:22:06.146677 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.146648    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nrpkt" event={"ID":"3ccfb319-8736-425b-bc66-486e3c249e88","Type":"ContainerStarted","Data":"b0cb6a67b7553430026f2ff97fc27a663b2cbcac6091656d4b103c3b2f1e0ebc"}
Jul 27 17:22:06.148269 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.148237    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4vhhn" event={"ID":"d34c6ca9-5a01-465e-bfba-2b9b8d468e94","Type":"ContainerStarted","Data":"08b3cf6e15beeae108fd5ed56721944dfb55a0f2e52c96b6dc997f9fcbb00bf0"}
Jul 27 17:22:06.150542 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.150516    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2hlk2" event={"ID":"e29ddd7d-78f5-4b66-ad2d-3e5010129212","Type":"ContainerStarted","Data":"b732b1004535109327b73caef9c34653c4646ad0f74b65e4ea16764820704e7f"}
Jul 27 17:22:06.150665 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.150569    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2hlk2" event={"ID":"e29ddd7d-78f5-4b66-ad2d-3e5010129212","Type":"ContainerStarted","Data":"29c2ff0e2da73be0585e43c4205d9598364a22e97a0ce8ece2c624328ed98b55"}
Jul 27 17:22:06.150665 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.150621    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:22:06.153175 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.152766    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79978b54d7-gkcbp" event={"ID":"ca6c17c3-2130-45c6-b429-43aac7e44c5a","Type":"ContainerStarted","Data":"4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1"}
Jul 27 17:22:06.153175 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.152814    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79978b54d7-gkcbp" event={"ID":"ca6c17c3-2130-45c6-b429-43aac7e44c5a","Type":"ContainerStarted","Data":"01d0abafcfc8890305efb0ecb74d59e9cc719c0b70df58ec2aa6e5d9f9ab7156"}
Jul 27 17:22:06.154483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.154451    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2" event={"ID":"0968b2c9-caca-4e07-ae42-5422e344e407","Type":"ContainerStarted","Data":"40c53f005a4ee774e8603b72b515f35d4c903f94780f615f453abfc92ef4d895"}
Jul 27 17:22:06.167432 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.167374    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75895d884b-kprp5" podStartSLOduration=2.36786703 podStartE2EDuration="10.167351965s" totalImagesPullingTime="7.799484935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:06.1667991 +0000 UTC m=+63.874947431" watchObservedRunningTime="2026-07-27 17:22:06.167351965 +0000 UTC m=+63.875500283"
Jul 27 17:22:06.209664 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.209597    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2hlk2" podStartSLOduration=2.41784631 podStartE2EDuration="10.209575481s" totalImagesPullingTime="7.791729171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:06.209159122 +0000 UTC m=+63.917307441" watchObservedRunningTime="2026-07-27 17:22:06.209575481 +0000 UTC m=+63.917723800"
Jul 27 17:22:06.210133 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.210102    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79978b54d7-gkcbp" podStartSLOduration=5.210091394 podStartE2EDuration="5.210091394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:06.187568867 +0000 UTC m=+63.895717186" watchObservedRunningTime="2026-07-27 17:22:06.210091394 +0000 UTC m=+63.918239712"
Jul 27 17:22:06.227328 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.226917    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4vhhn" podStartSLOduration=2.420625382 podStartE2EDuration="10.226891677s" totalImagesPullingTime="7.806266295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:06.226124415 +0000 UTC m=+63.934272733" watchObservedRunningTime="2026-07-27 17:22:06.226891677 +0000 UTC m=+63.935039997"
Jul 27 17:22:06.743281 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:06.743228    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:07.261411 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:07.261360    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:22:07.261586 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:07.261465    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:22:07.267764 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:07.267736    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:22:08.170858 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.170827    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:22:08.581617 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.581574    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:22:08.581844 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.581641    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:22:08.585059 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.585033    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 17:22:08.585222 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.585030    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 17:22:08.595224 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.595186    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5022fa8c-0fd5-4291-ab27-15b6ddae7ad2-original-pull-secret\") pod \"global-pull-secret-syncer-lc6cl\" (UID: \"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2\") " pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:22:08.595361 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.595197    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d37267a-ce1b-4238-ad69-34ada6577972-metrics-certs\") pod \"network-metrics-daemon-xk28w\" (UID: \"6d37267a-ce1b-4238-ad69-34ada6577972\") " pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:22:08.682651 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.682611    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsnr\" (UniqueName: \"kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr\") pod \"network-check-target-r8f9c\" (UID: \"0c54c59c-5570-4620-ae15-9fb594f16b2f\") " pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:22:08.686373 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.686347    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 17:22:08.696057 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.696029    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 17:22:08.706729 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.706673    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrsnr\" (UniqueName: \"kubernetes.io/projected/0c54c59c-5570-4620-ae15-9fb594f16b2f-kube-api-access-hrsnr\") pod \"network-check-target-r8f9c\" (UID: \"0c54c59c-5570-4620-ae15-9fb594f16b2f\") " pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:22:08.830158 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.830125    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lc6cl"
Jul 27 17:22:08.838884 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.838610    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfvk\""
Jul 27 17:22:08.846315 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.846292    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xk28w"
Jul 27 17:22:08.930611 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.930577    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dgczs\""
Jul 27 17:22:08.938241 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:08.938210    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:22:09.140348 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.140306    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xk28w"]
Jul 27 17:22:09.143443 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:09.143415    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d37267a_ce1b_4238_ad69_34ada6577972.slice/crio-1529d4c7f375ec4b075a9a3d68aec05e1cfa70acbd4f8b3fcb91538579de4c0a WatchSource:0}: Error finding container 1529d4c7f375ec4b075a9a3d68aec05e1cfa70acbd4f8b3fcb91538579de4c0a: Status 404 returned error can't find the container with id 1529d4c7f375ec4b075a9a3d68aec05e1cfa70acbd4f8b3fcb91538579de4c0a
Jul 27 17:22:09.151188 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.151100    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r8f9c"]
Jul 27 17:22:09.155138 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:09.154931    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c54c59c_5570_4620_ae15_9fb594f16b2f.slice/crio-b71112f30ce1a40964634f8afef85710a7a27e10edc43cad703f3a306f0f8e7f WatchSource:0}: Error finding container b71112f30ce1a40964634f8afef85710a7a27e10edc43cad703f3a306f0f8e7f: Status 404 returned error can't find the container with id b71112f30ce1a40964634f8afef85710a7a27e10edc43cad703f3a306f0f8e7f
Jul 27 17:22:09.169533 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.169487    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2" event={"ID":"0968b2c9-caca-4e07-ae42-5422e344e407","Type":"ContainerStarted","Data":"c7b24025ec30368acd29f7a30e3a46fd3f3de9391d65356e66e4aa5e662bf67a"}
Jul 27 17:22:09.169837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.169778    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2"
Jul 27 17:22:09.170934 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.170909    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lc6cl"]
Jul 27 17:22:09.171498 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.171435    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xk28w" event={"ID":"6d37267a-ce1b-4238-ad69-34ada6577972","Type":"ContainerStarted","Data":"1529d4c7f375ec4b075a9a3d68aec05e1cfa70acbd4f8b3fcb91538579de4c0a"}
Jul 27 17:22:09.173418 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.173384    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r8f9c" event={"ID":"0c54c59c-5570-4620-ae15-9fb594f16b2f","Type":"ContainerStarted","Data":"b71112f30ce1a40964634f8afef85710a7a27e10edc43cad703f3a306f0f8e7f"}
Jul 27 17:22:09.174856 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.174835    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2"
Jul 27 17:22:09.175202 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:09.175182    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5022fa8c_0fd5_4291_ab27_15b6ddae7ad2.slice/crio-d3c0b1f8a41bfa02821be5336e95c0bc419fc19e3c33ef866ce33de5c3a05736 WatchSource:0}: Error finding container d3c0b1f8a41bfa02821be5336e95c0bc419fc19e3c33ef866ce33de5c3a05736: Status 404 returned error can't find the container with id d3c0b1f8a41bfa02821be5336e95c0bc419fc19e3c33ef866ce33de5c3a05736
Jul 27 17:22:09.185915 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.185870    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qh2r2" podStartSLOduration=5.645941613 podStartE2EDuration="9.185853378s" totalImagesPullingTime="3.539911765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:09.185618958 +0000 UTC m=+66.893767275" watchObservedRunningTime="2026-07-27 17:22:09.185853378 +0000 UTC m=+66.894001695"
Jul 27 17:22:09.308087 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.308019    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:09.355206 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.355112    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"]
Jul 27 17:22:09.377221 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.377191    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"]
Jul 27 17:22:09.377373 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.377329    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.380257 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.380213    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 27 17:22:09.380257 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.380280    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 27 17:22:09.380257 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.380292    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 17:22:09.380531 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.380324    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tdc79\""
Jul 27 17:22:09.380531 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.380412    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 17:22:09.380601 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.380566    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 17:22:09.388011 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.387990    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24fb7cdb-2ad4-4e28-885c-42887a33e92b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.388117 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.388022    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c58ms\" (UniqueName: \"kubernetes.io/projected/24fb7cdb-2ad4-4e28-885c-42887a33e92b-kube-api-access-c58ms\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.388117 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.388047    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24fb7cdb-2ad4-4e28-885c-42887a33e92b-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.388117 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.388067    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/24fb7cdb-2ad4-4e28-885c-42887a33e92b-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.397898 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.397878    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:09.488595 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.488555    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24fb7cdb-2ad4-4e28-885c-42887a33e92b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.488595 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.488599    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c58ms\" (UniqueName: \"kubernetes.io/projected/24fb7cdb-2ad4-4e28-885c-42887a33e92b-kube-api-access-c58ms\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.488869 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.488627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24fb7cdb-2ad4-4e28-885c-42887a33e92b-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.488869 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.488644    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/24fb7cdb-2ad4-4e28-885c-42887a33e92b-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.488869 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:22:09.488784    2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 27 17:22:09.488869 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:22:09.488841    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24fb7cdb-2ad4-4e28-885c-42887a33e92b-prometheus-operator-tls podName:24fb7cdb-2ad4-4e28-885c-42887a33e92b nodeName:}" failed. No retries permitted until 2026-07-27 17:22:09.988824669 +0000 UTC m=+67.696972965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/24fb7cdb-2ad4-4e28-885c-42887a33e92b-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-jzqgn" (UID: "24fb7cdb-2ad4-4e28-885c-42887a33e92b") : secret "prometheus-operator-tls" not found
Jul 27 17:22:09.489356 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.489333    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24fb7cdb-2ad4-4e28-885c-42887a33e92b-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.491090 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.491065    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24fb7cdb-2ad4-4e28-885c-42887a33e92b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.498242 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.498210    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c58ms\" (UniqueName: \"kubernetes.io/projected/24fb7cdb-2ad4-4e28-885c-42887a33e92b-kube-api-access-c58ms\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.940330 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.940299    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:09.990840 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.990795    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/24fb7cdb-2ad4-4e28-885c-42887a33e92b-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:09.993809 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:09.993781    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/24fb7cdb-2ad4-4e28-885c-42887a33e92b-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-jzqgn\" (UID: \"24fb7cdb-2ad4-4e28-885c-42887a33e92b\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:10.179098 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:10.179063    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nrpkt" event={"ID":"3ccfb319-8736-425b-bc66-486e3c249e88","Type":"ContainerStarted","Data":"2cfcba749de9e8491f4485694481774fd52e9e62f01e8098624ba21a3e39b389"}
Jul 27 17:22:10.180059 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:10.180030    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lc6cl" event={"ID":"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2","Type":"ContainerStarted","Data":"d3c0b1f8a41bfa02821be5336e95c0bc419fc19e3c33ef866ce33de5c3a05736"}
Jul 27 17:22:10.286878 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:10.286792    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"
Jul 27 17:22:10.426734 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:10.426676    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-jzqgn"]
Jul 27 17:22:10.489845 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:10.489814    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24fb7cdb_2ad4_4e28_885c_42887a33e92b.slice/crio-615d0521c83a9c8f254d273b3dc0aed8c17c978d4e26ca9bc538e96e7f6782e3 WatchSource:0}: Error finding container 615d0521c83a9c8f254d273b3dc0aed8c17c978d4e26ca9bc538e96e7f6782e3: Status 404 returned error can't find the container with id 615d0521c83a9c8f254d273b3dc0aed8c17c978d4e26ca9bc538e96e7f6782e3
Jul 27 17:22:10.529114 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:10.529089    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:10.590278 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:10.590247    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:11.153857 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.153821    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:11.188431 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.187929    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xk28w" event={"ID":"6d37267a-ce1b-4238-ad69-34ada6577972","Type":"ContainerStarted","Data":"afabaea1bbfb7bbcca7a8ec63c2192380c46f72f0c337eb5cf64133a3bc5469f"}
Jul 27 17:22:11.188431 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.187989    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xk28w" event={"ID":"6d37267a-ce1b-4238-ad69-34ada6577972","Type":"ContainerStarted","Data":"cc0d6fdb9dfd6434f97bfaee09b6af9ceb9abadd7e7b0d594974576fba8652b2"}
Jul 27 17:22:11.190789 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.190728    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn" event={"ID":"24fb7cdb-2ad4-4e28-885c-42887a33e92b","Type":"ContainerStarted","Data":"615d0521c83a9c8f254d273b3dc0aed8c17c978d4e26ca9bc538e96e7f6782e3"}
Jul 27 17:22:11.213121 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.213014    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:11.303467 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.303354    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:11.373704 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.373644    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:11.877902 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.877149    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:11.877902 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.877226    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:11.884499 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.884469    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:11.904985 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:11.904916    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xk28w" podStartSLOduration=67.538866286 podStartE2EDuration="1m8.904893122s" totalImagesPullingTime="1.366026836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:11.20555566 +0000 UTC m=+68.913703982" watchObservedRunningTime="2026-07-27 17:22:11.904893122 +0000 UTC m=+69.613041442"
Jul 27 17:22:12.201551 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:12.201287    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:12.270073 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:12.270034    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75895d884b-kprp5"]
Jul 27 17:22:13.188210 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:13.188176    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:13.246848 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:13.246807    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:13.378203 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:13.378165    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:13.480070 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:13.479978    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:16.161892 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:16.161865    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2hlk2"
Jul 27 17:22:16.327797 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:16.327762    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:16.430045 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:16.429929    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:17.196582 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.196482    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:17.215736 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.215678    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nrpkt" event={"ID":"3ccfb319-8736-425b-bc66-486e3c249e88","Type":"ContainerStarted","Data":"a50971ea38b1200bb448f345ca5e27b921661b0b5f5dd4d2cc60499de881c239"}
Jul 27 17:22:17.217501 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.217465    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn" event={"ID":"24fb7cdb-2ad4-4e28-885c-42887a33e92b","Type":"ContainerStarted","Data":"c81151a65cfffec98bb4a685fd3e456396b576368d2711a94df756998efa78af"}
Jul 27 17:22:17.217649 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.217518    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn" event={"ID":"24fb7cdb-2ad4-4e28-885c-42887a33e92b","Type":"ContainerStarted","Data":"f89e23d86f2ae9984b7466aeca0353a588aa7ee944bf6f85fbc9156ae2091643"}
Jul 27 17:22:17.218820 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.218797    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r8f9c" event={"ID":"0c54c59c-5570-4620-ae15-9fb594f16b2f","Type":"ContainerStarted","Data":"6239912e5a56a560d2c7c1570771f644dd043c8e1d26b0947ec37e0fc5573b55"}
Jul 27 17:22:17.218944 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.218892    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:22:17.220202 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.220178    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lc6cl" event={"ID":"5022fa8c-0fd5-4291-ab27-15b6ddae7ad2","Type":"ContainerStarted","Data":"3f5255da073d533ae9435246229bf9c32dffc5abefd845e59795cfefface910e"}
Jul 27 17:22:17.237954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.237898    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nrpkt" podStartSLOduration=1.781573715 podStartE2EDuration="21.2378804s" totalImagesPullingTime="19.456306685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:17.236131527 +0000 UTC m=+74.944279845" watchObservedRunningTime="2026-07-27 17:22:17.2378804 +0000 UTC m=+74.946028769"
Jul 27 17:22:17.255679 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.255622    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-jzqgn" podStartSLOduration=1.901110426 podStartE2EDuration="8.255605235s" totalImagesPullingTime="6.354494809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:17.255076472 +0000 UTC m=+74.963224791" watchObservedRunningTime="2026-07-27 17:22:17.255605235 +0000 UTC m=+74.963753553"
Jul 27 17:22:17.273090 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.273039    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lc6cl" podStartSLOduration=66.577033056 podStartE2EDuration="1m14.273021328s" totalImagesPullingTime="7.695988272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:17.271958082 +0000 UTC m=+74.980106400" watchObservedRunningTime="2026-07-27 17:22:17.273021328 +0000 UTC m=+74.981169645"
Jul 27 17:22:17.925121 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.925088    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:17.981975 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:17.981943    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:18.451936 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.451901    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:18.636007 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.635978    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:18.672600 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.672540    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r8f9c" podStartSLOduration=67.958529351 podStartE2EDuration="1m15.672518662s" totalImagesPullingTime="7.713989311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:17.291724156 +0000 UTC m=+74.999872478" watchObservedRunningTime="2026-07-27 17:22:18.672518662 +0000 UTC m=+76.380666980"
Jul 27 17:22:18.673345 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.673326    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"]
Jul 27 17:22:18.675575 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.675560    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.680644 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.680617    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 27 17:22:18.680644 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.680632    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 27 17:22:18.681126 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.681102    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-s4nh6\""
Jul 27 17:22:18.681242 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.681144    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 27 17:22:18.690621 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.690564    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"]
Jul 27 17:22:18.703388 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.703311    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"]
Jul 27 17:22:18.705366 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.705350    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.708289 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.708268    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 27 17:22:18.708430 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.708368    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-95f4v\""
Jul 27 17:22:18.708532 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.708517    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 27 17:22:18.719770 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.719738    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"]
Jul 27 17:22:18.736136 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.736103    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mchtj"]
Jul 27 17:22:18.738510 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.738493    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.741324 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.741303    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 17:22:18.741471 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.741378    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 17:22:18.741471 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.741386    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 17:22:18.741471 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.741405    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k48sk\""
Jul 27 17:22:18.749318 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.749289    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:18.759705 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.759653    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.759705 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.759706    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7rzt\" (UniqueName: \"kubernetes.io/projected/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-api-access-h7rzt\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.759873 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.759740    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.759873 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.759792    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.759873 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.759823    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.759873 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.759847    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.760001 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.759890    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-accelerators-collector-config\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.760001 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.759934    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-textfile\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.760001 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.759968    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.760001 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.759990    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.760114 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.760038    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/923ea7a6-0b87-46f3-9f76-efcae55c4b60-metrics-client-ca\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.760114 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.760057    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.760114 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.760077    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-tls\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.760114 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.760103    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-wtmp\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.760230 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.760127    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.760230 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.760143    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/923ea7a6-0b87-46f3-9f76-efcae55c4b60-root\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.760230 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.760166    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5s77\" (UniqueName: \"kubernetes.io/projected/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-kube-api-access-j5s77\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.760230 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.760211    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/923ea7a6-0b87-46f3-9f76-efcae55c4b60-sys\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.760230 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.760228    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chpxd\" (UniqueName: \"kubernetes.io/projected/923ea7a6-0b87-46f3-9f76-efcae55c4b60-kube-api-access-chpxd\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.816323 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.816289    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:18.861495 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.861444    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5s77\" (UniqueName: \"kubernetes.io/projected/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-kube-api-access-j5s77\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.861680 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.861532    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/923ea7a6-0b87-46f3-9f76-efcae55c4b60-sys\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.861680 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.861556    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chpxd\" (UniqueName: \"kubernetes.io/projected/923ea7a6-0b87-46f3-9f76-efcae55c4b60-kube-api-access-chpxd\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.861680 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.861584    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.861680 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.861616    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7rzt\" (UniqueName: \"kubernetes.io/projected/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-api-access-h7rzt\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.861680 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.861637    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/923ea7a6-0b87-46f3-9f76-efcae55c4b60-sys\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.861680 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.861653    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.861999 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.861719    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.861999 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.861751    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.861999 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.861910    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.862158 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862000    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-accelerators-collector-config\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.862158 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862041    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-textfile\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.862158 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862073    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.862158 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862100    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.862158 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862134    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/923ea7a6-0b87-46f3-9f76-efcae55c4b60-metrics-client-ca\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.862580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862168    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.862580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862198    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-tls\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.862580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862230    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-wtmp\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.862580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862282    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.862580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862311    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/923ea7a6-0b87-46f3-9f76-efcae55c4b60-root\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.862580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862355    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-textfile\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.862580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862454    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.862580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862478    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.862580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862513    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/923ea7a6-0b87-46f3-9f76-efcae55c4b60-root\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.863053 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862737    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-wtmp\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.863053 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862810    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/923ea7a6-0b87-46f3-9f76-efcae55c4b60-metrics-client-ca\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.863053 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.862863    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-accelerators-collector-config\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.863053 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:22:18.862977    2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 27 17:22:18.863053 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:22:18.863050    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-tls podName:923ea7a6-0b87-46f3-9f76-efcae55c4b60 nodeName:}" failed. No retries permitted until 2026-07-27 17:22:19.363029198 +0000 UTC m=+77.071177494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-tls") pod "node-exporter-mchtj" (UID: "923ea7a6-0b87-46f3-9f76-efcae55c4b60") : secret "node-exporter-tls" not found
Jul 27 17:22:18.863311 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.863265    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.863474 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.863453    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.864556 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.864530    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.864664 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.864559    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.865061 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.865041    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.865240 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.865225    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.865480 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.865461    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.870504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.870480    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chpxd\" (UniqueName: \"kubernetes.io/projected/923ea7a6-0b87-46f3-9f76-efcae55c4b60-kube-api-access-chpxd\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:18.876869 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.876844    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5s77\" (UniqueName: \"kubernetes.io/projected/47880bcc-3ccc-4a21-8942-9a3a95bcbf31-kube-api-access-j5s77\") pod \"openshift-state-metrics-b8f49f5c6-lg5tp\" (UID: \"47880bcc-3ccc-4a21-8942-9a3a95bcbf31\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:18.877356 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.877332    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7rzt\" (UniqueName: \"kubernetes.io/projected/9856cdad-56b2-40a5-b31e-7ad5903ac0a2-kube-api-access-h7rzt\") pod \"kube-state-metrics-56d9f944b4-m4wht\" (UID: \"9856cdad-56b2-40a5-b31e-7ad5903ac0a2\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:18.896209 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.896179    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:18.984855 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:18.984768    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"
Jul 27 17:22:19.014609 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.014577    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"
Jul 27 17:22:19.032897 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.032817    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:19.125854 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.125824    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht"]
Jul 27 17:22:19.128175 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:19.128149    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9856cdad_56b2_40a5_b31e_7ad5903ac0a2.slice/crio-d299c962def8e277e16e88bf03cacac8b86545a7a3d1e7f17b7165dba7faafc2 WatchSource:0}: Error finding container d299c962def8e277e16e88bf03cacac8b86545a7a3d1e7f17b7165dba7faafc2: Status 404 returned error can't find the container with id d299c962def8e277e16e88bf03cacac8b86545a7a3d1e7f17b7165dba7faafc2
Jul 27 17:22:19.154474 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.154432    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp"]
Jul 27 17:22:19.158586 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:19.158557    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47880bcc_3ccc_4a21_8942_9a3a95bcbf31.slice/crio-ac0286be06ce5804acea1d65bc35451b73f9272d8c33297d07f1a47bdeb573c9 WatchSource:0}: Error finding container ac0286be06ce5804acea1d65bc35451b73f9272d8c33297d07f1a47bdeb573c9: Status 404 returned error can't find the container with id ac0286be06ce5804acea1d65bc35451b73f9272d8c33297d07f1a47bdeb573c9
Jul 27 17:22:19.230766 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.230729    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht" event={"ID":"9856cdad-56b2-40a5-b31e-7ad5903ac0a2","Type":"ContainerStarted","Data":"d299c962def8e277e16e88bf03cacac8b86545a7a3d1e7f17b7165dba7faafc2"}
Jul 27 17:22:19.232526 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.232498    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp" event={"ID":"47880bcc-3ccc-4a21-8942-9a3a95bcbf31","Type":"ContainerStarted","Data":"c257623e00d3e6a8eaa11747c94ef27199f9c8a82d0eb16d43649f954caa451c"}
Jul 27 17:22:19.232660 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.232560    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp" event={"ID":"47880bcc-3ccc-4a21-8942-9a3a95bcbf31","Type":"ContainerStarted","Data":"ac0286be06ce5804acea1d65bc35451b73f9272d8c33297d07f1a47bdeb573c9"}
Jul 27 17:22:19.337484 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.337451    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:19.365062 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.365015    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-tls\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:19.367652 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.367618    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/923ea7a6-0b87-46f3-9f76-efcae55c4b60-node-exporter-tls\") pod \"node-exporter-mchtj\" (UID: \"923ea7a6-0b87-46f3-9f76-efcae55c4b60\") " pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:19.438151 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.438117    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:19.649059 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.649013    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mchtj"
Jul 27 17:22:19.658585 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:19.658552    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod923ea7a6_0b87_46f3_9f76_efcae55c4b60.slice/crio-5e31c1376af3a1d1ff96423c9aaa39203573a89741310264e03b6ccef3b7b395 WatchSource:0}: Error finding container 5e31c1376af3a1d1ff96423c9aaa39203573a89741310264e03b6ccef3b7b395: Status 404 returned error can't find the container with id 5e31c1376af3a1d1ff96423c9aaa39203573a89741310264e03b6ccef3b7b395
Jul 27 17:22:19.783904 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.783861    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:19.935137 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.935052    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 17:22:19.938140 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.938123    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.942806 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.942781    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 27 17:22:19.942806 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.942779    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 27 17:22:19.943007 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.942814    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-727kq\""
Jul 27 17:22:19.943007 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.942808    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 27 17:22:19.943007 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.942781    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 27 17:22:19.943007 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.942922    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 27 17:22:19.943007 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.942928    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 27 17:22:19.943007 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.942993    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 27 17:22:19.943295 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.943186    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 27 17:22:19.943295 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.943239    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 27 17:22:19.949578 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.949543    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:19.970406 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970366    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970594 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970437    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-config-volume\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970594 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970483    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970594 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970507    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-web-config\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970594 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970533    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970594 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970566    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970602    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970623    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970641    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970662    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmgv4\" (UniqueName: \"kubernetes.io/projected/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-kube-api-access-jmgv4\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970705    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970735    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-config-out\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.970847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.970749    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:19.979814 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:19.979781    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 17:22:20.062019 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.061979    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:20.072156 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072125    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072174    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-config-volume\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072221    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072247    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-web-config\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072275    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072338 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072306    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072989 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072342    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072989 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072368    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072989 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072393    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072989 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072416    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmgv4\" (UniqueName: \"kubernetes.io/projected/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-kube-api-access-jmgv4\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072989 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072449    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072989 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072520    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-config-out\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.072989 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.072570    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.073485 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.073104    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.074172 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.074146    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.074916 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.074870    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.076256 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.076226    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.079777 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.079754    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.080034 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.080015    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-web-config\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.080837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.080174    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.080837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.080381    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.080837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.080481    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.080837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.080481    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-config-out\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.080837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.080514    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.081200 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.080908    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-config-volume\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.083282 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.083254    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmgv4\" (UniqueName: \"kubernetes.io/projected/e80ac313-5a55-4e61-be83-2b0cbdcf3e53-kube-api-access-jmgv4\") pod \"alertmanager-main-0\" (UID: \"e80ac313-5a55-4e61-be83-2b0cbdcf3e53\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.245386 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.245298    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp" event={"ID":"47880bcc-3ccc-4a21-8942-9a3a95bcbf31","Type":"ContainerStarted","Data":"d2f896817a2e5710c71405ecb9e52d6241125ae041049264f8557e0ff17dde6f"}
Jul 27 17:22:20.246637 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.246608    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mchtj" event={"ID":"923ea7a6-0b87-46f3-9f76-efcae55c4b60","Type":"ContainerStarted","Data":"5e31c1376af3a1d1ff96423c9aaa39203573a89741310264e03b6ccef3b7b395"}
Jul 27 17:22:20.248879 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.248857    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 27 17:22:20.621074 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.621038    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 27 17:22:20.624585 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:20.624550    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80ac313_5a55_4e61_be83_2b0cbdcf3e53.slice/crio-31fe5e434d7bee41b1beed5428a971e8090888be479f89f4a9a969468263b520 WatchSource:0}: Error finding container 31fe5e434d7bee41b1beed5428a971e8090888be479f89f4a9a969468263b520: Status 404 returned error can't find the container with id 31fe5e434d7bee41b1beed5428a971e8090888be479f89f4a9a969468263b520
Jul 27 17:22:20.713706 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.713658    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-749c48d54-scjkw"]
Jul 27 17:22:20.717651 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.717617    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.721281 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.720861    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 27 17:22:20.721663 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.721636    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 27 17:22:20.721827 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.721793    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 27 17:22:20.721827 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.721796    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-frs79qsequ70r\""
Jul 27 17:22:20.721936 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.721823    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-b4d9w\""
Jul 27 17:22:20.721936 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.721877    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 27 17:22:20.722030 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.721979    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 27 17:22:20.730106 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.730082    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-749c48d54-scjkw"]
Jul 27 17:22:20.779288 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.779216    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-grpc-tls\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.779288 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.779272    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.779494 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.779328    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-tls\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.779494 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.779360    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.779494 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.779448    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.779624 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.779503    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.779624 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.779563    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-metrics-client-ca\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.779624 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.779600    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8khk\" (UniqueName: \"kubernetes.io/projected/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-kube-api-access-k8khk\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.880568 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.880536    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.880744 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.880578    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-tls\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.880744 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.880633    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.880744 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.880668    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.880744 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.880720    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.880935 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.880762    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-metrics-client-ca\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.880935 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.880800    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8khk\" (UniqueName: \"kubernetes.io/projected/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-kube-api-access-k8khk\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.881267 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.881231    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-grpc-tls\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.881778 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.881743    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-metrics-client-ca\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.884035 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.883990    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.884986 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.884546    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.884986 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.884640    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.884986 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.884675    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-tls\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.884986 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.884932    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.885240 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.885121    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-secret-grpc-tls\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:20.890931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:20.890906    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8khk\" (UniqueName: \"kubernetes.io/projected/f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69-kube-api-access-k8khk\") pod \"thanos-querier-749c48d54-scjkw\" (UID: \"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69\") " pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:21.030052 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.029996    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:21.176501 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.176465    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-749c48d54-scjkw"]
Jul 27 17:22:21.179304 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:21.179273    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9ddb52d_29f0_4d65_83f7_9d8b8e96aa69.slice/crio-e274995a7a29579f8934c555fd61324d7f4ab0cc746ed49c6a5395c5452f3b73 WatchSource:0}: Error finding container e274995a7a29579f8934c555fd61324d7f4ab0cc746ed49c6a5395c5452f3b73: Status 404 returned error can't find the container with id e274995a7a29579f8934c555fd61324d7f4ab0cc746ed49c6a5395c5452f3b73
Jul 27 17:22:21.251163 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.251127    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80ac313-5a55-4e61-be83-2b0cbdcf3e53","Type":"ContainerStarted","Data":"31fe5e434d7bee41b1beed5428a971e8090888be479f89f4a9a969468263b520"}
Jul 27 17:22:21.253272 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.253244    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp" event={"ID":"47880bcc-3ccc-4a21-8942-9a3a95bcbf31","Type":"ContainerStarted","Data":"037c6b910566389429249fa6b3dd8f4743bdb4428f15130b87ca7ccf1858832d"}
Jul 27 17:22:21.254818 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.254792    2584 generic.go:356] "Generic (PLEG): container finished" podID="923ea7a6-0b87-46f3-9f76-efcae55c4b60" containerID="f8e51ca1cf3af4541b250a78c46577bacb57588fae107d3282ad50b9f43bad51" exitCode=0
Jul 27 17:22:21.254932 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.254876    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mchtj" event={"ID":"923ea7a6-0b87-46f3-9f76-efcae55c4b60","Type":"ContainerDied","Data":"f8e51ca1cf3af4541b250a78c46577bacb57588fae107d3282ad50b9f43bad51"}
Jul 27 17:22:21.255984 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.255966    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw" event={"ID":"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69","Type":"ContainerStarted","Data":"e274995a7a29579f8934c555fd61324d7f4ab0cc746ed49c6a5395c5452f3b73"}
Jul 27 17:22:21.276730 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.276663    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lg5tp" podStartSLOduration=2.110887349 podStartE2EDuration="3.276642275s" totalImagesPullingTime="1.165754926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:21.276287268 +0000 UTC m=+78.984435587" watchObservedRunningTime="2026-07-27 17:22:21.276642275 +0000 UTC m=+78.984790591"
Jul 27 17:22:21.513650 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.513603    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:21.618520 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.618482    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:21.814532 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.814475    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:21.915448 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:21.915415    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:22.015572 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.015532    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:22.150007 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.149667    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:22.251442 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.251406    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:22.268172 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.268133    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mchtj" event={"ID":"923ea7a6-0b87-46f3-9f76-efcae55c4b60","Type":"ContainerStarted","Data":"e0fdd443ddd91e3aa54eef49eb1b7757c0407f211d39a4fb34f573757f05f760"}
Jul 27 17:22:22.268301 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.268230    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mchtj" event={"ID":"923ea7a6-0b87-46f3-9f76-efcae55c4b60","Type":"ContainerStarted","Data":"06898f99fc8d981c1d3907e20038031cbe1fa99bafef5524e1a1f1ec9366f5b5"}
Jul 27 17:22:22.294117 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.294058    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mchtj" podStartSLOduration=2.905429224 podStartE2EDuration="4.2940399s" totalImagesPullingTime="1.31163858s" totalInitContainerRuntime="76.972096ms" isStatefulPod=true podCreationTimestamp="2026-07-27 17:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:22.292004813 +0000 UTC m=+80.000153131" watchObservedRunningTime="2026-07-27 17:22:22.2940399 +0000 UTC m=+80.002188217"
Jul 27 17:22:22.419851 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.415548    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:22.507544 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.507502    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:22.926474 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.926434    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-db6bff894-c4tpj"]
Jul 27 17:22:22.929577 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.929548    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:22.932661 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.932633    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 27 17:22:22.932840 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.932637    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 27 17:22:22.934180 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.934154    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-x4gq4\""
Jul 27 17:22:22.934180 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.934175    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 27 17:22:22.934373 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.934240    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 27 17:22:22.934541 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.934527    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9uo6o4c2mk48k\""
Jul 27 17:22:22.950949 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:22.950918    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-db6bff894-c4tpj"]
Jul 27 17:22:23.002224 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.002182    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c7ce4b16-6443-481b-b9cd-e96be4186dee-secret-metrics-server-tls\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.002410 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.002260    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg54r\" (UniqueName: \"kubernetes.io/projected/c7ce4b16-6443-481b-b9cd-e96be4186dee-kube-api-access-rg54r\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.002410 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.002293    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c7ce4b16-6443-481b-b9cd-e96be4186dee-audit-log\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.002410 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.002330    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7ce4b16-6443-481b-b9cd-e96be4186dee-client-ca-bundle\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.002410 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.002385    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ce4b16-6443-481b-b9cd-e96be4186dee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.002596 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.002425    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c7ce4b16-6443-481b-b9cd-e96be4186dee-metrics-server-audit-profiles\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.002596 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.002450    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c7ce4b16-6443-481b-b9cd-e96be4186dee-secret-metrics-server-client-certs\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.103182 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.103145    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ce4b16-6443-481b-b9cd-e96be4186dee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.103379 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.103209    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c7ce4b16-6443-481b-b9cd-e96be4186dee-metrics-server-audit-profiles\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.103379 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.103235    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c7ce4b16-6443-481b-b9cd-e96be4186dee-secret-metrics-server-client-certs\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.103379 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.103288    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c7ce4b16-6443-481b-b9cd-e96be4186dee-secret-metrics-server-tls\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.103379 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.103308    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg54r\" (UniqueName: \"kubernetes.io/projected/c7ce4b16-6443-481b-b9cd-e96be4186dee-kube-api-access-rg54r\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.103379 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.103332    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c7ce4b16-6443-481b-b9cd-e96be4186dee-audit-log\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.103379 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.103355    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7ce4b16-6443-481b-b9cd-e96be4186dee-client-ca-bundle\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.104219 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.104148    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c7ce4b16-6443-481b-b9cd-e96be4186dee-audit-log\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.104905 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.104876    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ce4b16-6443-481b-b9cd-e96be4186dee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.104995 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.104952    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c7ce4b16-6443-481b-b9cd-e96be4186dee-metrics-server-audit-profiles\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.106527 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.106498    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7ce4b16-6443-481b-b9cd-e96be4186dee-client-ca-bundle\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.106649 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.106598    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c7ce4b16-6443-481b-b9cd-e96be4186dee-secret-metrics-server-tls\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.106733 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.106716    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c7ce4b16-6443-481b-b9cd-e96be4186dee-secret-metrics-server-client-certs\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.120798 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.120677    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg54r\" (UniqueName: \"kubernetes.io/projected/c7ce4b16-6443-481b-b9cd-e96be4186dee-kube-api-access-rg54r\") pod \"metrics-server-db6bff894-c4tpj\" (UID: \"c7ce4b16-6443-481b-b9cd-e96be4186dee\") " pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.206266 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.206231    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:23.241349 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.241309    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:23.274069 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.274040    2584 generic.go:356] "Generic (PLEG): container finished" podID="e80ac313-5a55-4e61-be83-2b0cbdcf3e53" containerID="621203154bc00f49207fb02e22ff46446fd13ab9e453681289c84716567f44db" exitCode=0
Jul 27 17:22:23.274197 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.274124    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80ac313-5a55-4e61-be83-2b0cbdcf3e53","Type":"ContainerDied","Data":"621203154bc00f49207fb02e22ff46446fd13ab9e453681289c84716567f44db"}
Jul 27 17:22:23.276948 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.276915    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht" event={"ID":"9856cdad-56b2-40a5-b31e-7ad5903ac0a2","Type":"ContainerStarted","Data":"06929185e180c5a3a70d94a1a7d62fc0ad655a95619116eaf4ae462a5285a667"}
Jul 27 17:22:23.277064 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.276989    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht" event={"ID":"9856cdad-56b2-40a5-b31e-7ad5903ac0a2","Type":"ContainerStarted","Data":"ff842ccb2c3287e747b8fde268ba5e321dd2e3bce6208886a0ed0144bb2e3717"}
Jul 27 17:22:23.277064 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.277029    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht" event={"ID":"9856cdad-56b2-40a5-b31e-7ad5903ac0a2","Type":"ContainerStarted","Data":"64529aa6042f670b94821369901c6bd2a5d85a1258239c4c889f4ae758147ece"}
Jul 27 17:22:23.279427 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.279381    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw" event={"ID":"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69","Type":"ContainerStarted","Data":"48c9a732e334a4ce4d7df8e84824e4930eb3bb4baeb3928ff69c67fc936c7714"}
Jul 27 17:22:23.309627 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.307733    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:23.329483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.329411    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m4wht" podStartSLOduration=2.245182866 podStartE2EDuration="5.329386945s" totalImagesPullingTime="3.084204079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:23.327223603 +0000 UTC m=+81.035371920" watchObservedRunningTime="2026-07-27 17:22:23.329386945 +0000 UTC m=+81.037535270"
Jul 27 17:22:23.400162 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.399958    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-db6bff894-c4tpj"]
Jul 27 17:22:23.403399 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:23.403364    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ce4b16_6443_481b_b9cd_e96be4186dee.slice/crio-b13ee5e0eb675c255a51d07b19ab629e250828c16dccaa35fdd2bf9867362950 WatchSource:0}: Error finding container b13ee5e0eb675c255a51d07b19ab629e250828c16dccaa35fdd2bf9867362950: Status 404 returned error can't find the container with id b13ee5e0eb675c255a51d07b19ab629e250828c16dccaa35fdd2bf9867362950
Jul 27 17:22:23.514152 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.514109    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f84584f87-tgbc2"]
Jul 27 17:22:23.516236 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.516219    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.527966 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.527938    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:23.528329 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.528307    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"]
Jul 27 17:22:23.530623 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.530602    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"
Jul 27 17:22:23.531651 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.531625    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f84584f87-tgbc2"]
Jul 27 17:22:23.533922 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.533899    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 27 17:22:23.534025 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.533992    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-sg6c7\""
Jul 27 17:22:23.548746 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.548714    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"]
Jul 27 17:22:23.610958 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.610920    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkbnc\" (UniqueName: \"kubernetes.io/projected/07f38ee2-98fa-45bb-933a-4195f1d3dab6-kube-api-access-dkbnc\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.611151 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.610986    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-trusted-ca-bundle\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.611151 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.611017    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-config\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.611151 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.611067    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb2xk\" (UniqueName: \"kubernetes.io/projected/6900e0a3-9406-4bab-8863-542b0fd781ed-kube-api-access-zb2xk\") pod \"monitoring-plugin-68b7fc9574-4w8sp\" (UID: \"6900e0a3-9406-4bab-8863-542b0fd781ed\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"
Jul 27 17:22:23.611260 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.611161    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-service-ca\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.611260 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.611191    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6900e0a3-9406-4bab-8863-542b0fd781ed-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-4w8sp\" (UID: \"6900e0a3-9406-4bab-8863-542b0fd781ed\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"
Jul 27 17:22:23.611260 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.611216    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-serving-cert\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.611260 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.611235    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-oauth-config\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.611388 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.611271    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-oauth-serving-cert\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.642266 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.642166    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:23.712279 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.712249    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-oauth-serving-cert\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.712462 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.712309    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkbnc\" (UniqueName: \"kubernetes.io/projected/07f38ee2-98fa-45bb-933a-4195f1d3dab6-kube-api-access-dkbnc\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.712462 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.712351    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-trusted-ca-bundle\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.712462 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.712374    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-config\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.712640 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.712534    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb2xk\" (UniqueName: \"kubernetes.io/projected/6900e0a3-9406-4bab-8863-542b0fd781ed-kube-api-access-zb2xk\") pod \"monitoring-plugin-68b7fc9574-4w8sp\" (UID: \"6900e0a3-9406-4bab-8863-542b0fd781ed\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"
Jul 27 17:22:23.712640 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.712601    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-service-ca\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.712640 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.712633    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6900e0a3-9406-4bab-8863-542b0fd781ed-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-4w8sp\" (UID: \"6900e0a3-9406-4bab-8863-542b0fd781ed\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"
Jul 27 17:22:23.712818 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.712664    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-serving-cert\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.712818 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.712714    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-oauth-config\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.713153 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.713123    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-oauth-serving-cert\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.713275 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.713242    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-config\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.713467 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.713445    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-trusted-ca-bundle\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.713896 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.713874    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-service-ca\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.715729 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.715705    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6900e0a3-9406-4bab-8863-542b0fd781ed-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-4w8sp\" (UID: \"6900e0a3-9406-4bab-8863-542b0fd781ed\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"
Jul 27 17:22:23.716437 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.716413    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-serving-cert\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.716508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.716445    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-oauth-config\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.723720 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.723677    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb2xk\" (UniqueName: \"kubernetes.io/projected/6900e0a3-9406-4bab-8863-542b0fd781ed-kube-api-access-zb2xk\") pod \"monitoring-plugin-68b7fc9574-4w8sp\" (UID: \"6900e0a3-9406-4bab-8863-542b0fd781ed\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"
Jul 27 17:22:23.723720 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.723715    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkbnc\" (UniqueName: \"kubernetes.io/projected/07f38ee2-98fa-45bb-933a-4195f1d3dab6-kube-api-access-dkbnc\") pod \"console-5f84584f87-tgbc2\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") " pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.826328 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.826285    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:23.839915 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.839676    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"
Jul 27 17:22:23.958649 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.958600    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:23.992327 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:23.992247    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f84584f87-tgbc2"]
Jul 27 17:22:23.996925 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:23.996894    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f38ee2_98fa_45bb_933a_4195f1d3dab6.slice/crio-0933c1768f952b5f7798f07365a6ee2dee5d08cc112e3f08470ed49325cee535 WatchSource:0}: Error finding container 0933c1768f952b5f7798f07365a6ee2dee5d08cc112e3f08470ed49325cee535: Status 404 returned error can't find the container with id 0933c1768f952b5f7798f07365a6ee2dee5d08cc112e3f08470ed49325cee535
Jul 27 17:22:24.014965 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:24.014933    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"]
Jul 27 17:22:24.019890 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:24.019856    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6900e0a3_9406_4bab_8863_542b0fd781ed.slice/crio-ca80858e81d8f3baea57723845a40811fd6a4619062a3cb7f37dc392645e72b8 WatchSource:0}: Error finding container ca80858e81d8f3baea57723845a40811fd6a4619062a3cb7f37dc392645e72b8: Status 404 returned error can't find the container with id ca80858e81d8f3baea57723845a40811fd6a4619062a3cb7f37dc392645e72b8
Jul 27 17:22:24.130779 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:24.130651    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:24.203754 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:24.203570    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:24.291269 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:24.291230    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw" event={"ID":"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69","Type":"ContainerStarted","Data":"e746b362339da2ac62717a9fd84e0e8bf4121d4f23648baeae97b1fcd8ea069d"}
Jul 27 17:22:24.291437 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:24.291304    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw" event={"ID":"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69","Type":"ContainerStarted","Data":"27008e73da0a1eb93880fa32bc9ad41a10c1b80be342d8da7f3498aabfff5b1e"}
Jul 27 17:22:24.293583 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:24.293549    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp" event={"ID":"6900e0a3-9406-4bab-8863-542b0fd781ed","Type":"ContainerStarted","Data":"ca80858e81d8f3baea57723845a40811fd6a4619062a3cb7f37dc392645e72b8"}
Jul 27 17:22:24.295982 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:24.295954    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f84584f87-tgbc2" event={"ID":"07f38ee2-98fa-45bb-933a-4195f1d3dab6","Type":"ContainerStarted","Data":"924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a"}
Jul 27 17:22:24.296106 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:24.295999    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f84584f87-tgbc2" event={"ID":"07f38ee2-98fa-45bb-933a-4195f1d3dab6","Type":"ContainerStarted","Data":"0933c1768f952b5f7798f07365a6ee2dee5d08cc112e3f08470ed49325cee535"}
Jul 27 17:22:24.299013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:24.298971    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-db6bff894-c4tpj" event={"ID":"c7ce4b16-6443-481b-b9cd-e96be4186dee","Type":"ContainerStarted","Data":"b13ee5e0eb675c255a51d07b19ab629e250828c16dccaa35fdd2bf9867362950"}
Jul 27 17:22:24.317354 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:24.317298    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f84584f87-tgbc2" podStartSLOduration=1.31727736 podStartE2EDuration="1.31727736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:24.315785769 +0000 UTC m=+82.023934088" watchObservedRunningTime="2026-07-27 17:22:24.31727736 +0000 UTC m=+82.025425678"
Jul 27 17:22:25.027944 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.027901    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 17:22:25.033019 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.032988    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.037043 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.036404    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 27 17:22:25.037043 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.036590    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 27 17:22:25.037043 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.036631    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-41evq05nmil7d\""
Jul 27 17:22:25.037043 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.036894    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 27 17:22:25.037344 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.037104    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 27 17:22:25.037344 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.037211    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 27 17:22:25.037344 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.037296    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 27 17:22:25.037802 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.037784    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 27 17:22:25.037962 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.037911    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 27 17:22:25.038069 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.037839    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 27 17:22:25.038498 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.038313    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 27 17:22:25.038498 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.038346    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 27 17:22:25.038498 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.038375    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 27 17:22:25.038498 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.038346    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p7m85\""
Jul 27 17:22:25.050564 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.050532    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 27 17:22:25.053018 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.052983    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 27 17:22:25.061991 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.061961    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130154    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130209    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130265    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c1d5554-2844-4eb5-9239-fedc50739257-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130289    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c1d5554-2844-4eb5-9239-fedc50739257-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130312    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130352    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130376    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130410    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130434    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130515    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130566    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130662    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130743    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130769    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130792    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.131508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130815    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tckhn\" (UniqueName: \"kubernetes.io/projected/5c1d5554-2844-4eb5-9239-fedc50739257-kube-api-access-tckhn\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.132504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130848    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.132504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130887    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-config\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.132504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130910    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.132504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.130972    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.231544 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.231507    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c1d5554-2844-4eb5-9239-fedc50739257-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.231544 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.231548    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c1d5554-2844-4eb5-9239-fedc50739257-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.231824 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.231573    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.231824 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.231599    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.231824 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.231626    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.231824 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.231660    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.231824 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.231681    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.231824 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.231723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.231824 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.231751    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232213    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232298    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232327    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232346    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232368    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tckhn\" (UniqueName: \"kubernetes.io/projected/5c1d5554-2844-4eb5-9239-fedc50739257-kube-api-access-tckhn\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232394    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232426    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-config\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232448    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232484    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232509    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232532    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.232995    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.233009    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.233199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.233108    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.234202 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.234175    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.234432 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.234414    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.235398 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.235373    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.236191 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.236007    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.236953 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.236903    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c1d5554-2844-4eb5-9239-fedc50739257-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.237451 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.237416    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.237811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.237781    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c1d5554-2844-4eb5-9239-fedc50739257-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.238149 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.238108    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.238149 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.238126    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-config\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.238818 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.238771    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.239193 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.239167    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c1d5554-2844-4eb5-9239-fedc50739257-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.239358 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.239316    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.240027 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.239993    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.241015 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.240976    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.241096 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.241056    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.241500 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.241479    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c1d5554-2844-4eb5-9239-fedc50739257-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.242641 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.242615    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tckhn\" (UniqueName: \"kubernetes.io/projected/5c1d5554-2844-4eb5-9239-fedc50739257-kube-api-access-tckhn\") pod \"prometheus-k8s-0\" (UID: \"5c1d5554-2844-4eb5-9239-fedc50739257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.358220 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.356530    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:25.614925 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.614824    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 17:22:25.951117 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:22:25.951087    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1d5554_2844_4eb5_9239_fedc50739257.slice/crio-3d6690f74ba7d942436f79ef3c5ab710e2f80d94c2a5e4abe6054608d258dbcf WatchSource:0}: Error finding container 3d6690f74ba7d942436f79ef3c5ab710e2f80d94c2a5e4abe6054608d258dbcf: Status 404 returned error can't find the container with id 3d6690f74ba7d942436f79ef3c5ab710e2f80d94c2a5e4abe6054608d258dbcf
Jul 27 17:22:25.966879 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:25.966853    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:26.068100 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.068025    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:26.172296 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.172270    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:26.268873 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.268845    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:26.312456 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.312419    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw" event={"ID":"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69","Type":"ContainerStarted","Data":"d0fe94c65e27d8d990f72208122999edcb2c623e071a674b7672803818acd5d5"}
Jul 27 17:22:26.312800 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.312555    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw" event={"ID":"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69","Type":"ContainerStarted","Data":"cfd4a3c91f6d94534eaa72cee6b49431a7c4791bbc86ad8702168c046e5bc856"}
Jul 27 17:22:26.312800 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.312611    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw" event={"ID":"f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69","Type":"ContainerStarted","Data":"27d9702f1d29fa9a80222ed1ae1a9f6b7823886fb689ac4bb692ba58ba33ad02"}
Jul 27 17:22:26.312800 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.312642    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:26.314116 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.314092    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp" event={"ID":"6900e0a3-9406-4bab-8863-542b0fd781ed","Type":"ContainerStarted","Data":"b832f84e4539318c840df6a1a6abef47e57069270ea7cd730d0bfb8caca55e9f"}
Jul 27 17:22:26.314242 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.314210    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"
Jul 27 17:22:26.315952 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.315925    2584 generic.go:356] "Generic (PLEG): container finished" podID="5c1d5554-2844-4eb5-9239-fedc50739257" containerID="728d5c16eb938a87e791d4c9ca1ce8c0a599d1b513709f2688384a6a1bfc4531" exitCode=0
Jul 27 17:22:26.316072 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.316022    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c1d5554-2844-4eb5-9239-fedc50739257","Type":"ContainerDied","Data":"728d5c16eb938a87e791d4c9ca1ce8c0a599d1b513709f2688384a6a1bfc4531"}
Jul 27 17:22:26.316144 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.316091    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c1d5554-2844-4eb5-9239-fedc50739257","Type":"ContainerStarted","Data":"3d6690f74ba7d942436f79ef3c5ab710e2f80d94c2a5e4abe6054608d258dbcf"}
Jul 27 17:22:26.318509 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.318485    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-db6bff894-c4tpj" event={"ID":"c7ce4b16-6443-481b-b9cd-e96be4186dee","Type":"ContainerStarted","Data":"4d403d9d2a166908d0ec3aa7b8ca76e338ae9d3aebb88794f9526ed1168b1663"}
Jul 27 17:22:26.319967 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.319948    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp"
Jul 27 17:22:26.322042 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.322021    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80ac313-5a55-4e61-be83-2b0cbdcf3e53","Type":"ContainerStarted","Data":"cca2c9daa638fa328bbb14ee22142f86618d3d92ce664b3f2159a7e072403ad0"}
Jul 27 17:22:26.322123 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.322081    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80ac313-5a55-4e61-be83-2b0cbdcf3e53","Type":"ContainerStarted","Data":"226f1d5ef70c50b37e3dca2ff6a06c1901209c20a0621fd2d704d330a9d12994"}
Jul 27 17:22:26.322162 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.322123    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80ac313-5a55-4e61-be83-2b0cbdcf3e53","Type":"ContainerStarted","Data":"cd62dc9de26c5a9e5bce60821668378f7a33c0c77e98a324a68e86ef4bb05478"}
Jul 27 17:22:26.322195 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.322173    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80ac313-5a55-4e61-be83-2b0cbdcf3e53","Type":"ContainerStarted","Data":"5229dcb9ca0d24dd37265009130a48542bb6f2623f5a3e9635421e645c39d6e8"}
Jul 27 17:22:26.322231 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.322213    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80ac313-5a55-4e61-be83-2b0cbdcf3e53","Type":"ContainerStarted","Data":"8cbb218aec3394b82157eeb7bf0ab08ea9f40797b14a7b71556f38b26112c206"}
Jul 27 17:22:26.322274 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.322264    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80ac313-5a55-4e61-be83-2b0cbdcf3e53","Type":"ContainerStarted","Data":"82b7bdc049d12a22efaa25c366ed4466e5e0099832db14aa98f516bc3f3c0eae"}
Jul 27 17:22:26.336705 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.336642    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw" podStartSLOduration=2.393801083 podStartE2EDuration="6.336627115s" totalImagesPullingTime="3.942826032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:26.335634815 +0000 UTC m=+84.043783169" watchObservedRunningTime="2026-07-27 17:22:26.336627115 +0000 UTC m=+84.044775433"
Jul 27 17:22:26.354964 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.354910    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-db6bff894-c4tpj" podStartSLOduration=2.419073939 podStartE2EDuration="4.354893278s" totalImagesPullingTime="1.935819339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:26.354229904 +0000 UTC m=+84.062378247" watchObservedRunningTime="2026-07-27 17:22:26.354893278 +0000 UTC m=+84.063041596"
Jul 27 17:22:26.371024 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.370958    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4w8sp" podStartSLOduration=1.411291967 podStartE2EDuration="3.370941218s" totalImagesPullingTime="1.959649251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:26.370642352 +0000 UTC m=+84.078790672" watchObservedRunningTime="2026-07-27 17:22:26.370941218 +0000 UTC m=+84.079089536"
Jul 27 17:22:26.429057 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:26.428998    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.751035356 podStartE2EDuration="7.428980599s" totalImagesPullingTime="3.651916118s" totalInitContainerRuntime="26.029125ms" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:26.427280788 +0000 UTC m=+84.135429107" watchObservedRunningTime="2026-07-27 17:22:26.428980599 +0000 UTC m=+84.137128917"
Jul 27 17:22:30.347216 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:30.347171    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c1d5554-2844-4eb5-9239-fedc50739257","Type":"ContainerStarted","Data":"6b7a831c03bd47746463925950e8d0f439d69fdb3624d9ae60f0f1f743407745"}
Jul 27 17:22:30.347587 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:30.347268    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c1d5554-2844-4eb5-9239-fedc50739257","Type":"ContainerStarted","Data":"15c83ffe74f073d7bba3fc42a6a368b6e9aac46d28cb3ebb4520d6cf7c6a6950"}
Jul 27 17:22:30.347587 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:30.347295    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c1d5554-2844-4eb5-9239-fedc50739257","Type":"ContainerStarted","Data":"b2a1bd87d33d9a23867a087515943a4b7d648ee66a93c12a2e33f60b370d0fc6"}
Jul 27 17:22:30.347587 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:30.347318    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c1d5554-2844-4eb5-9239-fedc50739257","Type":"ContainerStarted","Data":"2622bef743e0aedf679da6231dbecea87bfcf6aee5abab5238b1912d92a6250a"}
Jul 27 17:22:30.347587 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:30.347342    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c1d5554-2844-4eb5-9239-fedc50739257","Type":"ContainerStarted","Data":"e5655120925e9985af21343b62714d18c49bda9d503c61077bdcc08664373977"}
Jul 27 17:22:30.347587 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:30.347364    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c1d5554-2844-4eb5-9239-fedc50739257","Type":"ContainerStarted","Data":"7c25321339d32dd7f33f34b0a7fe7df4c9de4af7e052d252b2e4eaf4540892f4"}
Jul 27 17:22:30.356908 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:30.356879    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:22:30.382427 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:30.382372    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.282042753 podStartE2EDuration="5.382352872s" totalImagesPullingTime="3.07059893s" totalInitContainerRuntime="29.711189ms" isStatefulPod=false podCreationTimestamp="2026-07-27 17:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:30.379520519 +0000 UTC m=+88.087668847" watchObservedRunningTime="2026-07-27 17:22:30.382352872 +0000 UTC m=+88.090501190"
Jul 27 17:22:31.450576 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:31.450515    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:31.522545 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:31.522508    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:32.304757 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:32.304719    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:32.335624 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:32.335595    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-749c48d54-scjkw"
Jul 27 17:22:32.373432 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:32.373397    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:33.826990 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:33.826944    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:33.827496 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:33.827017    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:33.831791 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:33.831767    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:34.368620 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:34.368594    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:22:34.427804 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:34.427765    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79978b54d7-gkcbp"]
Jul 27 17:22:34.569642 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:34.569609    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:34.684768 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:34.684657    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:34.785187 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:34.785156    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:37.300302 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:37.300259    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75895d884b-kprp5" podUID="ac688a9e-fd81-48da-bd08-28aa4775478c" containerName="console" containerID="cri-o://c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c" gracePeriod=15
Jul 27 17:22:38.112921 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.112890    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:22:38.266603 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.266508    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7w9j\" (UniqueName: \"kubernetes.io/projected/ac688a9e-fd81-48da-bd08-28aa4775478c-kube-api-access-z7w9j\") pod \"ac688a9e-fd81-48da-bd08-28aa4775478c\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") "
Jul 27 17:22:38.266603 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.266569    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-oauth-serving-cert\") pod \"ac688a9e-fd81-48da-bd08-28aa4775478c\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") "
Jul 27 17:22:38.266881 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.266616    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-console-config\") pod \"ac688a9e-fd81-48da-bd08-28aa4775478c\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") "
Jul 27 17:22:38.266881 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.266645    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-serving-cert\") pod \"ac688a9e-fd81-48da-bd08-28aa4775478c\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") "
Jul 27 17:22:38.266881 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.266680    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-oauth-config\") pod \"ac688a9e-fd81-48da-bd08-28aa4775478c\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") "
Jul 27 17:22:38.266881 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.266765    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-service-ca\") pod \"ac688a9e-fd81-48da-bd08-28aa4775478c\" (UID: \"ac688a9e-fd81-48da-bd08-28aa4775478c\") "
Jul 27 17:22:38.267076 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.267016    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ac688a9e-fd81-48da-bd08-28aa4775478c" (UID: "ac688a9e-fd81-48da-bd08-28aa4775478c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:22:38.267158 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.267130    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-console-config" (OuterVolumeSpecName: "console-config") pod "ac688a9e-fd81-48da-bd08-28aa4775478c" (UID: "ac688a9e-fd81-48da-bd08-28aa4775478c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:22:38.267228 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.267209    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-service-ca" (OuterVolumeSpecName: "service-ca") pod "ac688a9e-fd81-48da-bd08-28aa4775478c" (UID: "ac688a9e-fd81-48da-bd08-28aa4775478c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:22:38.269229 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.269199    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ac688a9e-fd81-48da-bd08-28aa4775478c" (UID: "ac688a9e-fd81-48da-bd08-28aa4775478c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:22:38.269334 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.269247    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac688a9e-fd81-48da-bd08-28aa4775478c-kube-api-access-z7w9j" (OuterVolumeSpecName: "kube-api-access-z7w9j") pod "ac688a9e-fd81-48da-bd08-28aa4775478c" (UID: "ac688a9e-fd81-48da-bd08-28aa4775478c"). InnerVolumeSpecName "kube-api-access-z7w9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:22:38.269334 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.269247    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ac688a9e-fd81-48da-bd08-28aa4775478c" (UID: "ac688a9e-fd81-48da-bd08-28aa4775478c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:22:38.367837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.367794    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7w9j\" (UniqueName: \"kubernetes.io/projected/ac688a9e-fd81-48da-bd08-28aa4775478c-kube-api-access-z7w9j\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:38.367837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.367826    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-oauth-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:38.367837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.367835    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-console-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:38.367837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.367844    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:38.368309 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.367853    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac688a9e-fd81-48da-bd08-28aa4775478c-console-oauth-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:38.368309 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.367861    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac688a9e-fd81-48da-bd08-28aa4775478c-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:38.384406 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.384372    2584 generic.go:356] "Generic (PLEG): container finished" podID="ac688a9e-fd81-48da-bd08-28aa4775478c" containerID="c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c" exitCode=0
Jul 27 17:22:38.384543 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.384418    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75895d884b-kprp5"
Jul 27 17:22:38.384543 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.384452    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75895d884b-kprp5" event={"ID":"ac688a9e-fd81-48da-bd08-28aa4775478c","Type":"ContainerDied","Data":"c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c"}
Jul 27 17:22:38.384543 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.384520    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75895d884b-kprp5" event={"ID":"ac688a9e-fd81-48da-bd08-28aa4775478c","Type":"ContainerDied","Data":"fc3a7f6e40023eba22e87a80dd9e6cd40014e9a03af8ec3b4655a31045c9b880"}
Jul 27 17:22:38.384543 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.384540    2584 scope.go:117] "RemoveContainer" containerID="c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c"
Jul 27 17:22:38.394146 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.394126    2584 scope.go:117] "RemoveContainer" containerID="c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c"
Jul 27 17:22:38.394475 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:22:38.394451    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c\": container with ID starting with c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c not found: ID does not exist" containerID="c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c"
Jul 27 17:22:38.394528 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.394486    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c"} err="failed to get container status \"c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c\": rpc error: code = NotFound desc = could not find container \"c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c\": container with ID starting with c3abc13add3b3b2522c6d9b405fceab44448797837ad9a252ca488b3191d411c not found: ID does not exist"
Jul 27 17:22:38.407869 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.407798    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75895d884b-kprp5"]
Jul 27 17:22:38.410626 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.410602    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75895d884b-kprp5"]
Jul 27 17:22:38.822478 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:38.822446    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac688a9e-fd81-48da-bd08-28aa4775478c" path="/var/lib/kubelet/pods/ac688a9e-fd81-48da-bd08-28aa4775478c/volumes"
Jul 27 17:22:41.158420 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:41.158380    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:41.254913 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:41.254871    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:43.242357 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:43.242315    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:43.242766 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:43.242388    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:22:48.225752 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:48.225717    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r8f9c"
Jul 27 17:22:52.644554 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:52.644511    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:52.746276 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:52.746238    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:52.847962 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:52.847928    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:22:59.453080 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.453037    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79978b54d7-gkcbp" podUID="ca6c17c3-2130-45c6-b429-43aac7e44c5a" containerName="console" containerID="cri-o://4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1" gracePeriod=15
Jul 27 17:22:59.708890 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.708823    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:22:59.767879 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.767002    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-service-ca\") pod \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") "
Jul 27 17:22:59.767879 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.767064    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nstkb\" (UniqueName: \"kubernetes.io/projected/ca6c17c3-2130-45c6-b429-43aac7e44c5a-kube-api-access-nstkb\") pod \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") "
Jul 27 17:22:59.767879 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.767099    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-config\") pod \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") "
Jul 27 17:22:59.767879 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.767136    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-trusted-ca-bundle\") pod \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") "
Jul 27 17:22:59.767879 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.767216    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-oauth-serving-cert\") pod \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") "
Jul 27 17:22:59.767879 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.767251    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-oauth-config\") pod \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") "
Jul 27 17:22:59.767879 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.767283    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-serving-cert\") pod \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\" (UID: \"ca6c17c3-2130-45c6-b429-43aac7e44c5a\") "
Jul 27 17:22:59.768940 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.768709    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-config" (OuterVolumeSpecName: "console-config") pod "ca6c17c3-2130-45c6-b429-43aac7e44c5a" (UID: "ca6c17c3-2130-45c6-b429-43aac7e44c5a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:22:59.769175 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.769034    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ca6c17c3-2130-45c6-b429-43aac7e44c5a" (UID: "ca6c17c3-2130-45c6-b429-43aac7e44c5a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:22:59.769587 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.769321    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-service-ca" (OuterVolumeSpecName: "service-ca") pod "ca6c17c3-2130-45c6-b429-43aac7e44c5a" (UID: "ca6c17c3-2130-45c6-b429-43aac7e44c5a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:22:59.769587 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.769577    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ca6c17c3-2130-45c6-b429-43aac7e44c5a" (UID: "ca6c17c3-2130-45c6-b429-43aac7e44c5a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:22:59.781551 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.781491    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ca6c17c3-2130-45c6-b429-43aac7e44c5a" (UID: "ca6c17c3-2130-45c6-b429-43aac7e44c5a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:22:59.781667 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.781549    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ca6c17c3-2130-45c6-b429-43aac7e44c5a" (UID: "ca6c17c3-2130-45c6-b429-43aac7e44c5a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:22:59.781667 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.781566    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca6c17c3-2130-45c6-b429-43aac7e44c5a-kube-api-access-nstkb" (OuterVolumeSpecName: "kube-api-access-nstkb") pod "ca6c17c3-2130-45c6-b429-43aac7e44c5a" (UID: "ca6c17c3-2130-45c6-b429-43aac7e44c5a"). InnerVolumeSpecName "kube-api-access-nstkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:22:59.868826 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.868786    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:59.868826 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.868816    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nstkb\" (UniqueName: \"kubernetes.io/projected/ca6c17c3-2130-45c6-b429-43aac7e44c5a-kube-api-access-nstkb\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:59.868826 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.868825    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:59.868826 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.868834    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-trusted-ca-bundle\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:59.869094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.868913    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca6c17c3-2130-45c6-b429-43aac7e44c5a-oauth-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:59.869094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.868924    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-oauth-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:22:59.869094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:22:59.868933    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca6c17c3-2130-45c6-b429-43aac7e44c5a-console-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:23:00.485770 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:00.485735    2584 generic.go:356] "Generic (PLEG): container finished" podID="ca6c17c3-2130-45c6-b429-43aac7e44c5a" containerID="4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1" exitCode=0
Jul 27 17:23:00.486247 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:00.485797    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79978b54d7-gkcbp"
Jul 27 17:23:00.486247 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:00.485808    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79978b54d7-gkcbp" event={"ID":"ca6c17c3-2130-45c6-b429-43aac7e44c5a","Type":"ContainerDied","Data":"4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1"}
Jul 27 17:23:00.486247 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:00.485849    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79978b54d7-gkcbp" event={"ID":"ca6c17c3-2130-45c6-b429-43aac7e44c5a","Type":"ContainerDied","Data":"01d0abafcfc8890305efb0ecb74d59e9cc719c0b70df58ec2aa6e5d9f9ab7156"}
Jul 27 17:23:00.486247 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:00.485868    2584 scope.go:117] "RemoveContainer" containerID="4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1"
Jul 27 17:23:00.496754 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:00.496732    2584 scope.go:117] "RemoveContainer" containerID="4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1"
Jul 27 17:23:00.497081 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:23:00.497059    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1\": container with ID starting with 4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1 not found: ID does not exist" containerID="4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1"
Jul 27 17:23:00.497163 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:00.497090    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1"} err="failed to get container status \"4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1\": rpc error: code = NotFound desc = could not find container \"4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1\": container with ID starting with 4044d9fcf7ceafdeff9cbb2ec332725cc5b2b8fbac7dbd32e1e3fda8130755c1 not found: ID does not exist"
Jul 27 17:23:00.509921 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:00.509888    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79978b54d7-gkcbp"]
Jul 27 17:23:00.513435 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:00.513410    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79978b54d7-gkcbp"]
Jul 27 17:23:00.822569 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:00.822536    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca6c17c3-2130-45c6-b429-43aac7e44c5a" path="/var/lib/kubelet/pods/ca6c17c3-2130-45c6-b429-43aac7e44c5a/volumes"
Jul 27 17:23:03.247818 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:03.247790    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:23:03.251958 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:03.251931    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-db6bff894-c4tpj"
Jul 27 17:23:25.357429 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:25.357390    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:23:25.377722 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:25.377656    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:23:25.610402 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:25.610313    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 17:23:38.085895 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:38.085850    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:23:38.288081 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:38.288043    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:23:45.240783 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:45.240746    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:23:48.036614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.036571    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8696f4b4b7-5jqpg"]
Jul 27 17:23:48.037078 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.036929    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca6c17c3-2130-45c6-b429-43aac7e44c5a" containerName="console"
Jul 27 17:23:48.037078 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.036942    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca6c17c3-2130-45c6-b429-43aac7e44c5a" containerName="console"
Jul 27 17:23:48.037078 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.036969    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac688a9e-fd81-48da-bd08-28aa4775478c" containerName="console"
Jul 27 17:23:48.037078 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.036974    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac688a9e-fd81-48da-bd08-28aa4775478c" containerName="console"
Jul 27 17:23:48.037078 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.037025    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca6c17c3-2130-45c6-b429-43aac7e44c5a" containerName="console"
Jul 27 17:23:48.037078 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.037034    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac688a9e-fd81-48da-bd08-28aa4775478c" containerName="console"
Jul 27 17:23:48.040029 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.040010    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.050875 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.050848    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8696f4b4b7-5jqpg"]
Jul 27 17:23:48.090471 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.090434    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:23:48.104805 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.104769    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-trusted-ca-bundle\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.104954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.104812    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-oauth-config\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.104954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.104829    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-serving-cert\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.104954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.104927    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-service-ca\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.105062 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.104976    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-config\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.105062 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.104997    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-oauth-serving-cert\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.105062 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.105029    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54wt7\" (UniqueName: \"kubernetes.io/projected/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-kube-api-access-54wt7\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.153410 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.153379    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:23:48.206165 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.206131    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-trusted-ca-bundle\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.206329 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.206169    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-oauth-config\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.206329 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.206193    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-serving-cert\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.206329 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.206235    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-service-ca\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.206329 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.206276    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-config\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.206329 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.206291    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-oauth-serving-cert\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.206329 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.206314    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54wt7\" (UniqueName: \"kubernetes.io/projected/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-kube-api-access-54wt7\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.207208 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.207179    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-service-ca\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.207327 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.207251    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-trusted-ca-bundle\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.207327 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.207311    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-config\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.207428 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.207398    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-oauth-serving-cert\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.208886 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.208869    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-oauth-config\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.209036 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.209019    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-serving-cert\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.215439 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.215415    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54wt7\" (UniqueName: \"kubernetes.io/projected/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-kube-api-access-54wt7\") pod \"console-8696f4b4b7-5jqpg\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") " pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.349599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.349554    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:48.485457 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.485433    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8696f4b4b7-5jqpg"]
Jul 27 17:23:48.488209 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:23:48.488181    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89cebc2_7ba2_4e35_ae8e_21f89870a27f.slice/crio-d3fb9b3724841a7e0d91bcfc14c172a9b120f2ecb1b75d4714e1c121d08dfec9 WatchSource:0}: Error finding container d3fb9b3724841a7e0d91bcfc14c172a9b120f2ecb1b75d4714e1c121d08dfec9: Status 404 returned error can't find the container with id d3fb9b3724841a7e0d91bcfc14c172a9b120f2ecb1b75d4714e1c121d08dfec9
Jul 27 17:23:48.690955 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.690863    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8696f4b4b7-5jqpg" event={"ID":"a89cebc2-7ba2-4e35-ae8e-21f89870a27f","Type":"ContainerStarted","Data":"7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94"}
Jul 27 17:23:48.690955 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.690916    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8696f4b4b7-5jqpg" event={"ID":"a89cebc2-7ba2-4e35-ae8e-21f89870a27f","Type":"ContainerStarted","Data":"d3fb9b3724841a7e0d91bcfc14c172a9b120f2ecb1b75d4714e1c121d08dfec9"}
Jul 27 17:23:48.708033 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:48.707942    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8696f4b4b7-5jqpg" podStartSLOduration=0.707920046 podStartE2EDuration="707.920046ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:23:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:23:48.707626818 +0000 UTC m=+166.415775136" watchObservedRunningTime="2026-07-27 17:23:48.707920046 +0000 UTC m=+166.416068365"
Jul 27 17:23:53.292681 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:53.292596    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:23:53.352157 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:53.352118    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:23:58.350508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:58.350461    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:58.351021 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:58.350542    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:58.355433 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:58.355406    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:58.736385 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:58.736291    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:23:58.784098 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:58.784063    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f84584f87-tgbc2"]
Jul 27 17:23:58.857464 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:58.857431    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b95f967bb-ptmvb"]
Jul 27 17:23:58.859866 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:58.859844    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:58.873220 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:58.873192    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b95f967bb-ptmvb"]
Jul 27 17:23:59.006168 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.006063    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-oauth-serving-cert\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.006168 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.006136    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-trusted-ca-bundle\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.006353 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.006177    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-console-config\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.006353 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.006273    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-serving-cert\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.006353 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.006303    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-oauth-config\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.006353 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.006319    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76qcl\" (UniqueName: \"kubernetes.io/projected/af864fee-581d-43c8-ae03-6a9a04e8a577-kube-api-access-76qcl\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.006353 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.006335    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-service-ca\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.107255 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.107215    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-serving-cert\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.107255 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.107257    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-oauth-config\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.107467 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.107280    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76qcl\" (UniqueName: \"kubernetes.io/projected/af864fee-581d-43c8-ae03-6a9a04e8a577-kube-api-access-76qcl\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.107467 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.107308    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-service-ca\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.107467 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.107420    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-oauth-serving-cert\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.107587 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.107489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-trusted-ca-bundle\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.107587 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.107540    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-console-config\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.108149 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.108123    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-oauth-serving-cert\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.108268 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.108152    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-console-config\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.108268 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.108168    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-service-ca\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.108601 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.108578    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-trusted-ca-bundle\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.110087 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.110049    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-serving-cert\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.110185 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.110089    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-oauth-config\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.116069 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.116046    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76qcl\" (UniqueName: \"kubernetes.io/projected/af864fee-581d-43c8-ae03-6a9a04e8a577-kube-api-access-76qcl\") pod \"console-5b95f967bb-ptmvb\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") " pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.169857 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.169818    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:23:59.239251 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.239216    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:23:59.307841 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.307814    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b95f967bb-ptmvb"]
Jul 27 17:23:59.310570 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:23:59.310531    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf864fee_581d_43c8_ae03_6a9a04e8a577.slice/crio-5813a473bc89b5e6708ec7ad639f6687a2d19a1869c7b9e1aae66b26abe96290 WatchSource:0}: Error finding container 5813a473bc89b5e6708ec7ad639f6687a2d19a1869c7b9e1aae66b26abe96290: Status 404 returned error can't find the container with id 5813a473bc89b5e6708ec7ad639f6687a2d19a1869c7b9e1aae66b26abe96290
Jul 27 17:23:59.738607 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.738517    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b95f967bb-ptmvb" event={"ID":"af864fee-581d-43c8-ae03-6a9a04e8a577","Type":"ContainerStarted","Data":"b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51"}
Jul 27 17:23:59.738607 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.738567    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b95f967bb-ptmvb" event={"ID":"af864fee-581d-43c8-ae03-6a9a04e8a577","Type":"ContainerStarted","Data":"5813a473bc89b5e6708ec7ad639f6687a2d19a1869c7b9e1aae66b26abe96290"}
Jul 27 17:23:59.759625 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.759265    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b95f967bb-ptmvb" podStartSLOduration=1.7592426159999999 podStartE2EDuration="1.759242616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:23:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:23:59.758673509 +0000 UTC m=+177.466821821" watchObservedRunningTime="2026-07-27 17:23:59.759242616 +0000 UTC m=+177.467390935"
Jul 27 17:23:59.832335 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:23:59.832297    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:05.263415 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:05.263373    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:05.323291 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:05.323253    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:09.170053 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:09.170000    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:24:09.170606 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:09.170088    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:24:09.174725 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:09.174672    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:24:09.785205 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:09.785176    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 17:24:09.835593 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:09.835558    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8696f4b4b7-5jqpg"]
Jul 27 17:24:09.938968 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:09.938935    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:10.039811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:10.039718    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:10.113803 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:10.113775    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:11.157010 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:11.156976    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:11.257782 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:11.257741    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:11.314304 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:11.314273    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:22.438764 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:22.438722    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:23.804469 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:23.804426    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f84584f87-tgbc2" podUID="07f38ee2-98fa-45bb-933a-4195f1d3dab6" containerName="console" containerID="cri-o://924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a" gracePeriod=15
Jul 27 17:24:24.061818 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.061793    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:24:24.157439 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.157384    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-serving-cert\") pod \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") "
Jul 27 17:24:24.157439 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.157449    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-oauth-config\") pod \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") "
Jul 27 17:24:24.157737 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.157495    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-service-ca\") pod \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") "
Jul 27 17:24:24.157737 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.157515    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkbnc\" (UniqueName: \"kubernetes.io/projected/07f38ee2-98fa-45bb-933a-4195f1d3dab6-kube-api-access-dkbnc\") pod \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") "
Jul 27 17:24:24.157737 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.157531    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-config\") pod \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") "
Jul 27 17:24:24.157737 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.157550    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-oauth-serving-cert\") pod \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") "
Jul 27 17:24:24.157737 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.157644    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-trusted-ca-bundle\") pod \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\" (UID: \"07f38ee2-98fa-45bb-933a-4195f1d3dab6\") "
Jul 27 17:24:24.157990 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.157922    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-service-ca" (OuterVolumeSpecName: "service-ca") pod "07f38ee2-98fa-45bb-933a-4195f1d3dab6" (UID: "07f38ee2-98fa-45bb-933a-4195f1d3dab6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:24:24.158160 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.158127    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-config" (OuterVolumeSpecName: "console-config") pod "07f38ee2-98fa-45bb-933a-4195f1d3dab6" (UID: "07f38ee2-98fa-45bb-933a-4195f1d3dab6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:24:24.158243 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.158166    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "07f38ee2-98fa-45bb-933a-4195f1d3dab6" (UID: "07f38ee2-98fa-45bb-933a-4195f1d3dab6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:24:24.158243 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.158198    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "07f38ee2-98fa-45bb-933a-4195f1d3dab6" (UID: "07f38ee2-98fa-45bb-933a-4195f1d3dab6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:24:24.159861 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.159835    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "07f38ee2-98fa-45bb-933a-4195f1d3dab6" (UID: "07f38ee2-98fa-45bb-933a-4195f1d3dab6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:24:24.159983 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.159917    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f38ee2-98fa-45bb-933a-4195f1d3dab6-kube-api-access-dkbnc" (OuterVolumeSpecName: "kube-api-access-dkbnc") pod "07f38ee2-98fa-45bb-933a-4195f1d3dab6" (UID: "07f38ee2-98fa-45bb-933a-4195f1d3dab6"). InnerVolumeSpecName "kube-api-access-dkbnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:24:24.160105 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.160086    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "07f38ee2-98fa-45bb-933a-4195f1d3dab6" (UID: "07f38ee2-98fa-45bb-933a-4195f1d3dab6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:24:24.258521 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.258474    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:24.258521 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.258516    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkbnc\" (UniqueName: \"kubernetes.io/projected/07f38ee2-98fa-45bb-933a-4195f1d3dab6-kube-api-access-dkbnc\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:24.258521 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.258527    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:24.258521 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.258536    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-oauth-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:24.258521 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.258544    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07f38ee2-98fa-45bb-933a-4195f1d3dab6-trusted-ca-bundle\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:24.258862 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.258552    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:24.258862 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.258560    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07f38ee2-98fa-45bb-933a-4195f1d3dab6-console-oauth-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:24.844594 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.844562    2584 generic.go:356] "Generic (PLEG): container finished" podID="07f38ee2-98fa-45bb-933a-4195f1d3dab6" containerID="924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a" exitCode=0
Jul 27 17:24:24.844997 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.844613    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f84584f87-tgbc2"
Jul 27 17:24:24.844997 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.844651    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f84584f87-tgbc2" event={"ID":"07f38ee2-98fa-45bb-933a-4195f1d3dab6","Type":"ContainerDied","Data":"924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a"}
Jul 27 17:24:24.844997 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.844725    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f84584f87-tgbc2" event={"ID":"07f38ee2-98fa-45bb-933a-4195f1d3dab6","Type":"ContainerDied","Data":"0933c1768f952b5f7798f07365a6ee2dee5d08cc112e3f08470ed49325cee535"}
Jul 27 17:24:24.844997 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.844747    2584 scope.go:117] "RemoveContainer" containerID="924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a"
Jul 27 17:24:24.853180 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.853163    2584 scope.go:117] "RemoveContainer" containerID="924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a"
Jul 27 17:24:24.853515 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:24:24.853493    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a\": container with ID starting with 924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a not found: ID does not exist" containerID="924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a"
Jul 27 17:24:24.853581 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.853529    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a"} err="failed to get container status \"924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a\": rpc error: code = NotFound desc = could not find container \"924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a\": container with ID starting with 924dfe2cbf79a83bdca5f97cd7cc889af26ccdfe23a0b51bd71d5b430d48036a not found: ID does not exist"
Jul 27 17:24:24.862734 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.862703    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f84584f87-tgbc2"]
Jul 27 17:24:24.866240 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:24.866210    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f84584f87-tgbc2"]
Jul 27 17:24:25.814415 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:25.814370    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:25.881371 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:25.881328    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:24:26.822311 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:26.822278    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07f38ee2-98fa-45bb-933a-4195f1d3dab6" path="/var/lib/kubelet/pods/07f38ee2-98fa-45bb-933a-4195f1d3dab6/volumes"
Jul 27 17:24:34.855722 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:34.855641    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8696f4b4b7-5jqpg" podUID="a89cebc2-7ba2-4e35-ae8e-21f89870a27f" containerName="console" containerID="cri-o://7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94" gracePeriod=15
Jul 27 17:24:35.104128 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.104105    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:24:35.258185 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258092    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-oauth-config\") pod \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") "
Jul 27 17:24:35.258361 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258204    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-config\") pod \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") "
Jul 27 17:24:35.258361 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258248    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-trusted-ca-bundle\") pod \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") "
Jul 27 17:24:35.258361 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258277    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54wt7\" (UniqueName: \"kubernetes.io/projected/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-kube-api-access-54wt7\") pod \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") "
Jul 27 17:24:35.258361 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258337    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-serving-cert\") pod \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") "
Jul 27 17:24:35.258562 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258365    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-service-ca\") pod \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") "
Jul 27 17:24:35.258562 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258391    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-oauth-serving-cert\") pod \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\" (UID: \"a89cebc2-7ba2-4e35-ae8e-21f89870a27f\") "
Jul 27 17:24:35.258779 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258738    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-config" (OuterVolumeSpecName: "console-config") pod "a89cebc2-7ba2-4e35-ae8e-21f89870a27f" (UID: "a89cebc2-7ba2-4e35-ae8e-21f89870a27f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:24:35.258779 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258745    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a89cebc2-7ba2-4e35-ae8e-21f89870a27f" (UID: "a89cebc2-7ba2-4e35-ae8e-21f89870a27f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:24:35.258890 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258829    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-service-ca" (OuterVolumeSpecName: "service-ca") pod "a89cebc2-7ba2-4e35-ae8e-21f89870a27f" (UID: "a89cebc2-7ba2-4e35-ae8e-21f89870a27f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:24:35.258890 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.258862    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a89cebc2-7ba2-4e35-ae8e-21f89870a27f" (UID: "a89cebc2-7ba2-4e35-ae8e-21f89870a27f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:24:35.260595 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.260574    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a89cebc2-7ba2-4e35-ae8e-21f89870a27f" (UID: "a89cebc2-7ba2-4e35-ae8e-21f89870a27f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:24:35.261083 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.261055    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a89cebc2-7ba2-4e35-ae8e-21f89870a27f" (UID: "a89cebc2-7ba2-4e35-ae8e-21f89870a27f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:24:35.261179 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.261106    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-kube-api-access-54wt7" (OuterVolumeSpecName: "kube-api-access-54wt7") pod "a89cebc2-7ba2-4e35-ae8e-21f89870a27f" (UID: "a89cebc2-7ba2-4e35-ae8e-21f89870a27f"). InnerVolumeSpecName "kube-api-access-54wt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:24:35.359053 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.359019    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:35.359053 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.359050    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:35.359053 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.359063    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-oauth-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:35.359294 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.359077    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-oauth-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:35.359294 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.359090    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-console-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:35.359294 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.359102    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-trusted-ca-bundle\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:35.359294 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.359113    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54wt7\" (UniqueName: \"kubernetes.io/projected/a89cebc2-7ba2-4e35-ae8e-21f89870a27f-kube-api-access-54wt7\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:24:35.891817 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.891778    2584 generic.go:356] "Generic (PLEG): container finished" podID="a89cebc2-7ba2-4e35-ae8e-21f89870a27f" containerID="7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94" exitCode=0
Jul 27 17:24:35.892300 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.891827    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8696f4b4b7-5jqpg"
Jul 27 17:24:35.892300 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.891877    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8696f4b4b7-5jqpg" event={"ID":"a89cebc2-7ba2-4e35-ae8e-21f89870a27f","Type":"ContainerDied","Data":"7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94"}
Jul 27 17:24:35.892300 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.891931    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8696f4b4b7-5jqpg" event={"ID":"a89cebc2-7ba2-4e35-ae8e-21f89870a27f","Type":"ContainerDied","Data":"d3fb9b3724841a7e0d91bcfc14c172a9b120f2ecb1b75d4714e1c121d08dfec9"}
Jul 27 17:24:35.892300 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.891957    2584 scope.go:117] "RemoveContainer" containerID="7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94"
Jul 27 17:24:35.901088 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.901072    2584 scope.go:117] "RemoveContainer" containerID="7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94"
Jul 27 17:24:35.901379 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:24:35.901361    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94\": container with ID starting with 7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94 not found: ID does not exist" containerID="7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94"
Jul 27 17:24:35.901441 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.901393    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94"} err="failed to get container status \"7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94\": rpc error: code = NotFound desc = could not find container \"7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94\": container with ID starting with 7c5aa75a2c9d56ec7169ac6e7a2b83d444b9ad17d697fcfdec94aa6d821e8c94 not found: ID does not exist"
Jul 27 17:24:35.913404 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.913372    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8696f4b4b7-5jqpg"]
Jul 27 17:24:35.917237 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:35.917208    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8696f4b4b7-5jqpg"]
Jul 27 17:24:36.822622 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:24:36.822588    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a89cebc2-7ba2-4e35-ae8e-21f89870a27f" path="/var/lib/kubelet/pods/a89cebc2-7ba2-4e35-ae8e-21f89870a27f/volumes"
Jul 27 17:25:16.791756 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:16.791715    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:18.383061 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:18.383024    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:43.722170 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:43.722131    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:44.687803 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:44.687756    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:47.349184 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:47.349139    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:47.419023 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:47.418983    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:48.550977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:48.550932    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:48.651774 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:48.651740    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:49.453883 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:49.453840    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:49.586568 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:49.586535    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:51.632026 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:51.631993    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:52.047483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:52.047402    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:54.139735 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:54.139705    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:54.200469 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:54.200426    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:25:57.341632 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.341598    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f7l7w"]
Jul 27 17:25:57.342145 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.342072    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f38ee2-98fa-45bb-933a-4195f1d3dab6" containerName="console"
Jul 27 17:25:57.342145 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.342091    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f38ee2-98fa-45bb-933a-4195f1d3dab6" containerName="console"
Jul 27 17:25:57.342145 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.342106    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a89cebc2-7ba2-4e35-ae8e-21f89870a27f" containerName="console"
Jul 27 17:25:57.342145 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.342112    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a89cebc2-7ba2-4e35-ae8e-21f89870a27f" containerName="console"
Jul 27 17:25:57.342294 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.342179    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a89cebc2-7ba2-4e35-ae8e-21f89870a27f" containerName="console"
Jul 27 17:25:57.342294 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.342188    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f38ee2-98fa-45bb-933a-4195f1d3dab6" containerName="console"
Jul 27 17:25:57.344848 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.344832    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w"
Jul 27 17:25:57.347804 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.347783    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 27 17:25:57.347931 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.347890    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qpl2z\""
Jul 27 17:25:57.349197 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.349180    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 27 17:25:57.361219 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.361196    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f7l7w"]
Jul 27 17:25:57.485466 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.485433    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/093224b4-1637-4909-95c3-aaf882579a51-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f7l7w\" (UID: \"093224b4-1637-4909-95c3-aaf882579a51\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w"
Jul 27 17:25:57.485466 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.485477    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tzxk\" (UniqueName: \"kubernetes.io/projected/093224b4-1637-4909-95c3-aaf882579a51-kube-api-access-7tzxk\") pod \"cert-manager-webhook-6486ffd997-f7l7w\" (UID: \"093224b4-1637-4909-95c3-aaf882579a51\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w"
Jul 27 17:25:57.586930 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.586887    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/093224b4-1637-4909-95c3-aaf882579a51-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f7l7w\" (UID: \"093224b4-1637-4909-95c3-aaf882579a51\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w"
Jul 27 17:25:57.586930 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.586937    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tzxk\" (UniqueName: \"kubernetes.io/projected/093224b4-1637-4909-95c3-aaf882579a51-kube-api-access-7tzxk\") pod \"cert-manager-webhook-6486ffd997-f7l7w\" (UID: \"093224b4-1637-4909-95c3-aaf882579a51\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w"
Jul 27 17:25:57.597160 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.597084    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/093224b4-1637-4909-95c3-aaf882579a51-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f7l7w\" (UID: \"093224b4-1637-4909-95c3-aaf882579a51\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w"
Jul 27 17:25:57.597295 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.597197    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tzxk\" (UniqueName: \"kubernetes.io/projected/093224b4-1637-4909-95c3-aaf882579a51-kube-api-access-7tzxk\") pod \"cert-manager-webhook-6486ffd997-f7l7w\" (UID: \"093224b4-1637-4909-95c3-aaf882579a51\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w"
Jul 27 17:25:57.662590 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.662551    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w"
Jul 27 17:25:57.796353 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:57.796320    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f7l7w"]
Jul 27 17:25:57.799472 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:25:57.799438    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod093224b4_1637_4909_95c3_aaf882579a51.slice/crio-ed37235402b938a0b9ed3f08754f113c5ef5cad8d40a16f855d06abf9f1e1215 WatchSource:0}: Error finding container ed37235402b938a0b9ed3f08754f113c5ef5cad8d40a16f855d06abf9f1e1215: Status 404 returned error can't find the container with id ed37235402b938a0b9ed3f08754f113c5ef5cad8d40a16f855d06abf9f1e1215
Jul 27 17:25:58.220346 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:25:58.220307    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w" event={"ID":"093224b4-1637-4909-95c3-aaf882579a51","Type":"ContainerStarted","Data":"ed37235402b938a0b9ed3f08754f113c5ef5cad8d40a16f855d06abf9f1e1215"}
Jul 27 17:26:00.124696 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:00.124661    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:00.224796 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:00.224763    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:00.293396 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:00.293358    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:01.236735 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:01.236635    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w" event={"ID":"093224b4-1637-4909-95c3-aaf882579a51","Type":"ContainerStarted","Data":"9587327d24da1d715a59dba07667a6b2f46128f35aaf67de65e457d89314c803"}
Jul 27 17:26:01.237100 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:01.236746    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w"
Jul 27 17:26:01.256160 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:01.256110    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w" podStartSLOduration=1.181335235 podStartE2EDuration="4.256093897s" totalImagesPullingTime="3.074758662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:01.255662387 +0000 UTC m=+298.963810704" watchObservedRunningTime="2026-07-27 17:26:01.256093897 +0000 UTC m=+298.964242215"
Jul 27 17:26:02.356090 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.356057    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"]
Jul 27 17:26:02.359309 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.359292    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"
Jul 27 17:26:02.362320 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.362297    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:26:02.363778 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.363761    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-fxs99\""
Jul 27 17:26:02.363889 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.363796    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 27 17:26:02.369461 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.369435    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"]
Jul 27 17:26:02.436183 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.436143    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c166456-677b-4b37-876b-1c150f50cd98-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7gcc6\" (UID: \"1c166456-677b-4b37-876b-1c150f50cd98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"
Jul 27 17:26:02.436362 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.436194    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc7zn\" (UniqueName: \"kubernetes.io/projected/1c166456-677b-4b37-876b-1c150f50cd98-kube-api-access-vc7zn\") pod \"openshift-lws-operator-fd8ccff4c-7gcc6\" (UID: \"1c166456-677b-4b37-876b-1c150f50cd98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"
Jul 27 17:26:02.537615 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.537557    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c166456-677b-4b37-876b-1c150f50cd98-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7gcc6\" (UID: \"1c166456-677b-4b37-876b-1c150f50cd98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"
Jul 27 17:26:02.537821 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.537636    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc7zn\" (UniqueName: \"kubernetes.io/projected/1c166456-677b-4b37-876b-1c150f50cd98-kube-api-access-vc7zn\") pod \"openshift-lws-operator-fd8ccff4c-7gcc6\" (UID: \"1c166456-677b-4b37-876b-1c150f50cd98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"
Jul 27 17:26:02.538060 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.538037    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c166456-677b-4b37-876b-1c150f50cd98-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7gcc6\" (UID: \"1c166456-677b-4b37-876b-1c150f50cd98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"
Jul 27 17:26:02.547263 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.547242    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc7zn\" (UniqueName: \"kubernetes.io/projected/1c166456-677b-4b37-876b-1c150f50cd98-kube-api-access-vc7zn\") pod \"openshift-lws-operator-fd8ccff4c-7gcc6\" (UID: \"1c166456-677b-4b37-876b-1c150f50cd98\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"
Jul 27 17:26:02.671744 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.671556    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-fxs99\""
Jul 27 17:26:02.678542 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.678515    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"
Jul 27 17:26:02.683268 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.683242    2584 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 17:26:02.700783 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.700749    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:02.802749 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.802711    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:02.831935 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:26:02.821080    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c166456_677b_4b37_876b_1c150f50cd98.slice/crio-65d659ec51707384ba229a7baea4c856e439c780c4ad60a21be7ed00eeb84b8a WatchSource:0}: Error finding container 65d659ec51707384ba229a7baea4c856e439c780c4ad60a21be7ed00eeb84b8a: Status 404 returned error can't find the container with id 65d659ec51707384ba229a7baea4c856e439c780c4ad60a21be7ed00eeb84b8a
Jul 27 17:26:02.831935 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:02.822827    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6"]
Jul 27 17:26:03.246752 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:03.246718    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6" event={"ID":"1c166456-677b-4b37-876b-1c150f50cd98","Type":"ContainerStarted","Data":"65d659ec51707384ba229a7baea4c856e439c780c4ad60a21be7ed00eeb84b8a"}
Jul 27 17:26:07.243820 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:07.243786    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f7l7w"
Jul 27 17:26:09.277480 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:09.277429    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6" event={"ID":"1c166456-677b-4b37-876b-1c150f50cd98","Type":"ContainerStarted","Data":"98ae54e6076447dbb4d3b7308acdb79e570ff58e41f3bb37c701a54a347c6254"}
Jul 27 17:26:09.294514 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:09.294457    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7gcc6" podStartSLOduration=1.504250359 podStartE2EDuration="7.294438641s" totalImagesPullingTime="5.790188282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:09.293633496 +0000 UTC m=+307.001781814" watchObservedRunningTime="2026-07-27 17:26:09.294438641 +0000 UTC m=+307.002586959"
Jul 27 17:26:09.652151 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:09.652111    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:09.747023 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:09.746982    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:10.001917 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.001830    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:10.098601 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.098570    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:10.098796 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.098743    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jgcth"]
Jul 27 17:26:10.102314 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.102294    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jgcth"
Jul 27 17:26:10.105202 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.105181    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-lcbxd\""
Jul 27 17:26:10.114164 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.114134    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jgcth"]
Jul 27 17:26:10.212305 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.212265    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0f4eb41-3466-45c2-bdd5-bc716a84d118-bound-sa-token\") pod \"cert-manager-55b4d954d5-jgcth\" (UID: \"d0f4eb41-3466-45c2-bdd5-bc716a84d118\") " pod="cert-manager/cert-manager-55b4d954d5-jgcth"
Jul 27 17:26:10.212305 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.212310    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c6pn\" (UniqueName: \"kubernetes.io/projected/d0f4eb41-3466-45c2-bdd5-bc716a84d118-kube-api-access-4c6pn\") pod \"cert-manager-55b4d954d5-jgcth\" (UID: \"d0f4eb41-3466-45c2-bdd5-bc716a84d118\") " pod="cert-manager/cert-manager-55b4d954d5-jgcth"
Jul 27 17:26:10.312907 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.312872    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0f4eb41-3466-45c2-bdd5-bc716a84d118-bound-sa-token\") pod \"cert-manager-55b4d954d5-jgcth\" (UID: \"d0f4eb41-3466-45c2-bdd5-bc716a84d118\") " pod="cert-manager/cert-manager-55b4d954d5-jgcth"
Jul 27 17:26:10.313316 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.312918    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c6pn\" (UniqueName: \"kubernetes.io/projected/d0f4eb41-3466-45c2-bdd5-bc716a84d118-kube-api-access-4c6pn\") pod \"cert-manager-55b4d954d5-jgcth\" (UID: \"d0f4eb41-3466-45c2-bdd5-bc716a84d118\") " pod="cert-manager/cert-manager-55b4d954d5-jgcth"
Jul 27 17:26:10.322645 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.322605    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0f4eb41-3466-45c2-bdd5-bc716a84d118-bound-sa-token\") pod \"cert-manager-55b4d954d5-jgcth\" (UID: \"d0f4eb41-3466-45c2-bdd5-bc716a84d118\") " pod="cert-manager/cert-manager-55b4d954d5-jgcth"
Jul 27 17:26:10.322820 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.322709    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c6pn\" (UniqueName: \"kubernetes.io/projected/d0f4eb41-3466-45c2-bdd5-bc716a84d118-kube-api-access-4c6pn\") pod \"cert-manager-55b4d954d5-jgcth\" (UID: \"d0f4eb41-3466-45c2-bdd5-bc716a84d118\") " pod="cert-manager/cert-manager-55b4d954d5-jgcth"
Jul 27 17:26:10.411989 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.411960    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jgcth"
Jul 27 17:26:10.547791 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:10.547678    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jgcth"]
Jul 27 17:26:10.550297 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:26:10.550265    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0f4eb41_3466_45c2_bdd5_bc716a84d118.slice/crio-3db10f5a4e7c1e18ac08a8cee2712f853b4bc4f8a4d16585fae09b4f10c17d19 WatchSource:0}: Error finding container 3db10f5a4e7c1e18ac08a8cee2712f853b4bc4f8a4d16585fae09b4f10c17d19: Status 404 returned error can't find the container with id 3db10f5a4e7c1e18ac08a8cee2712f853b4bc4f8a4d16585fae09b4f10c17d19
Jul 27 17:26:11.288208 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:11.288174    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jgcth" event={"ID":"d0f4eb41-3466-45c2-bdd5-bc716a84d118","Type":"ContainerStarted","Data":"a478cd06ec67413eed4b472b30eb7276b359816a60f46923d7938dbe5635f702"}
Jul 27 17:26:11.288208 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:11.288215    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jgcth" event={"ID":"d0f4eb41-3466-45c2-bdd5-bc716a84d118","Type":"ContainerStarted","Data":"3db10f5a4e7c1e18ac08a8cee2712f853b4bc4f8a4d16585fae09b4f10c17d19"}
Jul 27 17:26:11.309430 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:11.309377    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-jgcth" podStartSLOduration=1.309361495 podStartE2EDuration="1.309361495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:11.308193494 +0000 UTC m=+309.016341811" watchObservedRunningTime="2026-07-27 17:26:11.309361495 +0000 UTC m=+309.017509813"
Jul 27 17:26:11.698570 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:11.698532    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:23.851925 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:23.851885    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:23.942563 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:23.942523    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:24.358145 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.358102    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:24.598094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.598059    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:24.713590 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.713502    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:24.813677 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.813644    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"]
Jul 27 17:26:24.817063 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.817041    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:24.820844 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.820826    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 17:26:24.820981 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.820863    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 17:26:24.821230 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.821207    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 17:26:24.821670 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.821469    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 17:26:24.822662 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.822640    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nqddh\""
Jul 27 17:26:24.838160 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.838130    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"]
Jul 27 17:26:24.940116 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.940085    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-tmp\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:24.940537 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.940127    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:24.940537 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.940264    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsvrg\" (UniqueName: \"kubernetes.io/projected/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-kube-api-access-nsvrg\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:24.940537 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.940300    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:24.940537 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:24.940330    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-manifests\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.009756 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.009627    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:25.041542 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.041500    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsvrg\" (UniqueName: \"kubernetes.io/projected/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-kube-api-access-nsvrg\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.041542 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.041540    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.041860 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.041563    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-manifests\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.041860 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.041714    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-tmp\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.041860 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.041756    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.042002 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.041981    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-manifests\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.044140 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.044116    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-tmp\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.044387 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.044370    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.044465 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.044441    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.054709 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.054660    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsvrg\" (UniqueName: \"kubernetes.io/projected/b6cd0e43-2bf0-4a3a-b409-290e146ff66d-kube-api-access-nsvrg\") pod \"opendatahub-operator-controller-manager-746b8d4d47-98thz\" (UID: \"b6cd0e43-2bf0-4a3a-b409-290e146ff66d\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.110323 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.110290    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:25.128515 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.128479    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:25.210416 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.210386    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:25.275063 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.274974    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"]
Jul 27 17:26:25.278605 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:26:25.278580    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6cd0e43_2bf0_4a3a_b409_290e146ff66d.slice/crio-01924aa34523273b7d20796f752f3a5fed4fa17eafc5fedbc0b1c5364b9b5763 WatchSource:0}: Error finding container 01924aa34523273b7d20796f752f3a5fed4fa17eafc5fedbc0b1c5364b9b5763: Status 404 returned error can't find the container with id 01924aa34523273b7d20796f752f3a5fed4fa17eafc5fedbc0b1c5364b9b5763
Jul 27 17:26:25.280581 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.280558    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:26:25.313313 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.313279    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:25.348350 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.348319    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz" event={"ID":"b6cd0e43-2bf0-4a3a-b409-290e146ff66d","Type":"ContainerStarted","Data":"01924aa34523273b7d20796f752f3a5fed4fa17eafc5fedbc0b1c5364b9b5763"}
Jul 27 17:26:25.415395 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.415361    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:25.520804 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.520769    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:25.612613 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:25.612586    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:29.374132 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:29.374096    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz" event={"ID":"b6cd0e43-2bf0-4a3a-b409-290e146ff66d","Type":"ContainerStarted","Data":"9ae0fd9b1dd0a516d2e74974220f067a19291498747e7f7dc963803003c4eac1"}
Jul 27 17:26:30.380005 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:30.379971    2584 generic.go:356] "Generic (PLEG): container finished" podID="b6cd0e43-2bf0-4a3a-b409-290e146ff66d" containerID="9ae0fd9b1dd0a516d2e74974220f067a19291498747e7f7dc963803003c4eac1" exitCode=0
Jul 27 17:26:30.380409 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:30.380057    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz" event={"ID":"b6cd0e43-2bf0-4a3a-b409-290e146ff66d","Type":"ContainerDied","Data":"9ae0fd9b1dd0a516d2e74974220f067a19291498747e7f7dc963803003c4eac1"}
Jul 27 17:26:31.387037 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:31.387000    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz" event={"ID":"b6cd0e43-2bf0-4a3a-b409-290e146ff66d","Type":"ContainerStarted","Data":"fa4b0159f0d4a791ef320f0dfe5bc78bd3fa91ac9f2bce7899fdcd24d60921c5"}
Jul 27 17:26:31.387441 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:31.387078    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:31.410888 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:31.410842    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz" podStartSLOduration=2.27725445 podStartE2EDuration="7.410825555s" totalImagesPullingTime="4.097657927s" totalInitContainerRuntime="1.035913178s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:31.409226511 +0000 UTC m=+329.117374830" watchObservedRunningTime="2026-07-27 17:26:31.410825555 +0000 UTC m=+329.118973872"
Jul 27 17:26:42.396304 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:42.396269    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-98thz"
Jul 27 17:26:43.944065 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:43.944032    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:44.086103 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:44.086066    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:44.491740 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:44.491706    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:44.778116 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:44.778024    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:44.868583 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:44.868549    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:44.987428 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:44.987393    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:45.166091 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:45.166055    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:45.600106 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:45.600069    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:45.855097 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:45.855018    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:46.040081 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:46.040053    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:46.152113 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:46.152022    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:47.445941 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.445900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:47.506427 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.506392    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"]
Jul 27 17:26:47.515826 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.515801    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.518870 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.518844    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-np22g\""
Jul 27 17:26:47.520218 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.520189    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 27 17:26:47.520347 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.520228    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 27 17:26:47.520347 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.520297    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 27 17:26:47.527199 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.527170    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"]
Jul 27 17:26:47.537013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.536975    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:47.642975 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.642936    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-manager-config\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.643151 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.642987    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-cert\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.643151 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.643061    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-metrics-cert\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.643227 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.643144    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zch2s\" (UniqueName: \"kubernetes.io/projected/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-kube-api-access-zch2s\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.744662 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.744573    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-manager-config\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.744662 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.744628    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-cert\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.744662 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.744669    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-metrics-cert\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.744900 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.744796    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zch2s\" (UniqueName: \"kubernetes.io/projected/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-kube-api-access-zch2s\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.745311 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.745276    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-manager-config\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.747429 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.747400    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-metrics-cert\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.747548 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.747448    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-cert\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.760068 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.760036    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zch2s\" (UniqueName: \"kubernetes.io/projected/9a3c5fbc-a3b4-491a-95c6-fefbd9059dff-kube-api-access-zch2s\") pod \"lws-controller-manager-76849bb8b4-l8qt7\" (UID: \"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.766413 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.766384    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:47.826977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.826935    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:47.994206 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:47.994175    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"]
Jul 27 17:26:47.997520 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:26:47.997493    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a3c5fbc_a3b4_491a_95c6_fefbd9059dff.slice/crio-56859e5b43bc481c6caa84368dcb0beed13a167c709b19aa411f1949da499823 WatchSource:0}: Error finding container 56859e5b43bc481c6caa84368dcb0beed13a167c709b19aa411f1949da499823: Status 404 returned error can't find the container with id 56859e5b43bc481c6caa84368dcb0beed13a167c709b19aa411f1949da499823
Jul 27 17:26:48.466162 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:48.466131    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7" event={"ID":"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff","Type":"ContainerStarted","Data":"56859e5b43bc481c6caa84368dcb0beed13a167c709b19aa411f1949da499823"}
Jul 27 17:26:50.308556 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.308530    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:50.388026 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.387956    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:50.479751 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.479703    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7" event={"ID":"9a3c5fbc-a3b4-491a-95c6-fefbd9059dff","Type":"ContainerStarted","Data":"4cb9f12eae0d0323cce25dabb7274c97a8a652d0db5e8fdf3d063b9808187c71"}
Jul 27 17:26:50.479950 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.479913    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:26:50.500302 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.500186    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7" podStartSLOduration=1.249468559 podStartE2EDuration="3.50016264s" totalImagesPullingTime="2.250694081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:50.49893062 +0000 UTC m=+348.207078937" watchObservedRunningTime="2026-07-27 17:26:50.50016264 +0000 UTC m=+348.208310959"
Jul 27 17:26:50.625515 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.625475    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:50.717792 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.717756    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"]
Jul 27 17:26:50.720203 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.720184    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.723553 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.723528    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 17:26:50.723735 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.723681    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 17:26:50.723818 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.723737    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 27 17:26:50.723818 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.723793    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-d6vx5\""
Jul 27 17:26:50.734373 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.734347    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"]
Jul 27 17:26:50.794937 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.794834    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:50.878816 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.878783    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btp77\" (UniqueName: \"kubernetes.io/projected/d2c01468-60be-450a-900e-3d9a6a43ccb0-kube-api-access-btp77\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.878994 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.878841    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c01468-60be-450a-900e-3d9a6a43ccb0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.878994 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.878863    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.878994 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.878955    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.879094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.879000    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.879094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.879024    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.879094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.879048    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.879094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.879075    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.879217 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.879097    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.980530 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.980490    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btp77\" (UniqueName: \"kubernetes.io/projected/d2c01468-60be-450a-900e-3d9a6a43ccb0-kube-api-access-btp77\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.980727 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.980550    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c01468-60be-450a-900e-3d9a6a43ccb0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.980780 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.980727    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.980841 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.980815    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.980887 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.980842    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.980887 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.980859    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.980887 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.980876    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.981096 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.980894    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.981096 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.980932    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.981254 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.981118    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.981254 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.981201    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.981420 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.981385    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.981567 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.981436    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c01468-60be-450a-900e-3d9a6a43ccb0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.981829 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.981631    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.983457 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.983435    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.983551 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.983491    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.989978 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.989957    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c01468-60be-450a-900e-3d9a6a43ccb0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.990287 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.990256    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btp77\" (UniqueName: \"kubernetes.io/projected/d2c01468-60be-450a-900e-3d9a6a43ccb0-kube-api-access-btp77\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx\" (UID: \"d2c01468-60be-450a-900e-3d9a6a43ccb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:50.999871 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:50.999843    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:51.031258 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:51.031213    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:51.188209 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:51.188040    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"]
Jul 27 17:26:51.190907 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:26:51.190867    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c01468_60be_450a_900e_3d9a6a43ccb0.slice/crio-d930c85b36cf828eba02f42c536d319c1865c2cfcbd5bc0b8eeb30a7d3c3340f WatchSource:0}: Error finding container d930c85b36cf828eba02f42c536d319c1865c2cfcbd5bc0b8eeb30a7d3c3340f: Status 404 returned error can't find the container with id d930c85b36cf828eba02f42c536d319c1865c2cfcbd5bc0b8eeb30a7d3c3340f
Jul 27 17:26:51.490432 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:51.490333    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx" event={"ID":"d2c01468-60be-450a-900e-3d9a6a43ccb0","Type":"ContainerStarted","Data":"d930c85b36cf828eba02f42c536d319c1865c2cfcbd5bc0b8eeb30a7d3c3340f"}
Jul 27 17:26:54.186037 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.185990    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:26:54.186478 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.186096    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:26:54.186478 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.186152    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:26:54.341768 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.341728    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"]
Jul 27 17:26:54.345900 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.345867    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.362267 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.362236    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 27 17:26:54.362530 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.362517    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v6bwz\""
Jul 27 17:26:54.362831 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.362817    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 27 17:26:54.363819 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.363805    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 27 17:26:54.388423 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.388387    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"]
Jul 27 17:26:54.510117 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.509979    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx" event={"ID":"d2c01468-60be-450a-900e-3d9a6a43ccb0","Type":"ContainerStarted","Data":"469abd041e3c721df7eee763271458020bc5392fac3bd5deb153d6e83d2a51e8"}
Jul 27 17:26:54.517301 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.517271    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.517481 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.517323    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.517481 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.517393    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26bbff3a-415e-4487-93a6-9d51305ba00e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.517596 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.517495    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26bbff3a-415e-4487-93a6-9d51305ba00e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.517596 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.517532    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.517596 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.517590    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.517730 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.517611    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7nk8\" (UniqueName: \"kubernetes.io/projected/26bbff3a-415e-4487-93a6-9d51305ba00e-kube-api-access-w7nk8\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.537991 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.537940    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx" podStartSLOduration=1.545252058 podStartE2EDuration="4.537923043s" totalImagesPullingTime="2.992670985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:54.536715761 +0000 UTC m=+352.244864075" watchObservedRunningTime="2026-07-27 17:26:54.537923043 +0000 UTC m=+352.246071361"
Jul 27 17:26:54.618262 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.618215    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26bbff3a-415e-4487-93a6-9d51305ba00e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.618560 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.618314    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.618560 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.618372    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.618560 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.618395    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7nk8\" (UniqueName: \"kubernetes.io/projected/26bbff3a-415e-4487-93a6-9d51305ba00e-kube-api-access-w7nk8\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.618560 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.618443    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.619063 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.619036    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.619201 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.619074    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26bbff3a-415e-4487-93a6-9d51305ba00e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.619201 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.619104    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.620957 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.620935    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26bbff3a-415e-4487-93a6-9d51305ba00e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.621173 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.621153    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.621241 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.621194    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.621542 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.621527    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26bbff3a-415e-4487-93a6-9d51305ba00e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.628413 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.628389    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26bbff3a-415e-4487-93a6-9d51305ba00e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.628643 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.628623    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7nk8\" (UniqueName: \"kubernetes.io/projected/26bbff3a-415e-4487-93a6-9d51305ba00e-kube-api-access-w7nk8\") pod \"istiod-openshift-gateway-5677ccb49b-zgrlm\" (UID: \"26bbff3a-415e-4487-93a6-9d51305ba00e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.656743 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.656704    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:54.805747 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:54.805720    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"]
Jul 27 17:26:54.808041 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:26:54.808013    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26bbff3a_415e_4487_93a6_9d51305ba00e.slice/crio-c3cb31a6c965921dfe587bb0e12a6964aae2c16855a7ca366bc34c335eb5a3de WatchSource:0}: Error finding container c3cb31a6c965921dfe587bb0e12a6964aae2c16855a7ca366bc34c335eb5a3de: Status 404 returned error can't find the container with id c3cb31a6c965921dfe587bb0e12a6964aae2c16855a7ca366bc34c335eb5a3de
Jul 27 17:26:55.031970 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:55.031931    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:55.036566 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:55.036536    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:55.517220 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:55.517171    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm" event={"ID":"26bbff3a-415e-4487-93a6-9d51305ba00e","Type":"ContainerStarted","Data":"c3cb31a6c965921dfe587bb0e12a6964aae2c16855a7ca366bc34c335eb5a3de"}
Jul 27 17:26:55.517751 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:55.517442    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:55.518486 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:55.518455    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx"
Jul 27 17:26:56.945580 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:56.945546    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:57.072058 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:57.072024    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:57.684814 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:57.684755    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:58.098616 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:58.098579    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:26:58.098940 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:58.098650    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:26:58.275799 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:58.275765    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:58.482615 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:58.482494    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:58.536145 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:58.536108    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm" event={"ID":"26bbff3a-415e-4487-93a6-9d51305ba00e","Type":"ContainerStarted","Data":"b31dca1892a01f8ef042028d654146704dbcdfcad37992157f47c523e3dc7238"}
Jul 27 17:26:58.536371 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:58.536218    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:26:58.563472 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:58.563426    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm" podStartSLOduration=1.275249842 podStartE2EDuration="4.563409399s" totalImagesPullingTime="3.288159557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:58.561627955 +0000 UTC m=+356.269776287" watchObservedRunningTime="2026-07-27 17:26:58.563409399 +0000 UTC m=+356.271557717"
Jul 27 17:26:58.568115 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:58.568071    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:26:59.542809 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:26:59.542783    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zgrlm"
Jul 27 17:27:01.492661 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:27:01.492627    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-l8qt7"
Jul 27 17:27:45.453930 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:27:45.453895    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:27:45.551474 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:27:45.551434    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:27:45.628333 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:27:45.628307    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:27:57.495224 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:27:57.495188    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:14.650248 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:14.650195    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:18.107306 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:18.107267    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:19.655820 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:19.655786    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:20.052916 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:20.052815    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:21.085832 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:21.085802    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:21.182678 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:21.182644    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:22.649220 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:22.649184    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:23.710481 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:23.710444    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:25.056020 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:25.055983    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:27.719865 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:27.719831    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:28.869748 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:28.869680    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:28.971859 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:28.971820    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:29.161910 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:29.161823    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:29.307489 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:29.307455    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:38.971288 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:38.971250    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:39.065357 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:39.065321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:39.163919 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:39.163888    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:41.951141 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:41.951103    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:42.065227 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:42.065193    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:44.383541 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:44.383510    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:44.483530 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:44.483498    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:50.073538 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:50.073495    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:50.135328 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:50.135293    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:52.435750 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:52.435711    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:28:52.585448 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:28:52.585418    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:00.488561 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:00.488523    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:00.563074 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:00.563031    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:01.077971 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:01.077941    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:01.181005 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:01.180957    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:04.484078 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:04.484036    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:04.584875 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:04.584839    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:05.793634 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:05.793591    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:05.898200 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:05.898166    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:06.082934 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:06.082900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:06.183628 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:06.183598    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:11.478542 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:11.478509    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:18.792010 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:18.791970    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:18.870138 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:18.870095    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:23.443443 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:23.443403    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:23.544153 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:23.544119    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:25.241636 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:25.241595    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:25.363309 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:25.363276    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:25.433820 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:25.433785    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:26.102807 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:26.102777    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:26.204307 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:26.204278    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:46.335512 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:46.335467    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:46.434153 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:46.434112    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:46.821453 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:46.821417    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:58.686498 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:58.686410    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:58.999710 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:29:58.999589    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:05.202664 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.202624    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:05.219632 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.219599    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-blr6c"]
Jul 27 17:30:05.222144 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.222123    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:05.225366 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.225343    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 17:30:05.225837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.225818    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 17:30:05.226835 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.226814    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bps2r\""
Jul 27 17:30:05.234288 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.234258    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-blr6c"]
Jul 27 17:30:05.323160 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.323131    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04f8702c-7f90-4b82-b9b1-d137ab6eb324-webhook-cert\") pod \"maas-controller-57b4cbfdf4-blr6c\" (UID: \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\") " pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:05.323160 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.323161    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwnpr\" (UniqueName: \"kubernetes.io/projected/04f8702c-7f90-4b82-b9b1-d137ab6eb324-kube-api-access-fwnpr\") pod \"maas-controller-57b4cbfdf4-blr6c\" (UID: \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\") " pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:05.424086 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.424057    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04f8702c-7f90-4b82-b9b1-d137ab6eb324-webhook-cert\") pod \"maas-controller-57b4cbfdf4-blr6c\" (UID: \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\") " pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:05.424086 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.424088    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwnpr\" (UniqueName: \"kubernetes.io/projected/04f8702c-7f90-4b82-b9b1-d137ab6eb324-kube-api-access-fwnpr\") pod \"maas-controller-57b4cbfdf4-blr6c\" (UID: \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\") " pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:05.424324 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:30:05.424209    2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 27 17:30:05.424324 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:30:05.424287    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f8702c-7f90-4b82-b9b1-d137ab6eb324-webhook-cert podName:04f8702c-7f90-4b82-b9b1-d137ab6eb324 nodeName:}" failed. No retries permitted until 2026-07-27 17:30:05.924270249 +0000 UTC m=+543.632418545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/04f8702c-7f90-4b82-b9b1-d137ab6eb324-webhook-cert") pod "maas-controller-57b4cbfdf4-blr6c" (UID: "04f8702c-7f90-4b82-b9b1-d137ab6eb324") : secret "maas-controller-webhook-cert" not found
Jul 27 17:30:05.433368 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.433341    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwnpr\" (UniqueName: \"kubernetes.io/projected/04f8702c-7f90-4b82-b9b1-d137ab6eb324-kube-api-access-fwnpr\") pod \"maas-controller-57b4cbfdf4-blr6c\" (UID: \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\") " pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:05.616239 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.616207    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:05.681066 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.681027    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:05.929134 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.929028    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04f8702c-7f90-4b82-b9b1-d137ab6eb324-webhook-cert\") pod \"maas-controller-57b4cbfdf4-blr6c\" (UID: \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\") " pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:05.931632 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:05.931600    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04f8702c-7f90-4b82-b9b1-d137ab6eb324-webhook-cert\") pod \"maas-controller-57b4cbfdf4-blr6c\" (UID: \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\") " pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:06.133917 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:06.133861    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:06.268708 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:06.268663    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-blr6c"]
Jul 27 17:30:06.270671 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:30:06.270641    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04f8702c_7f90_4b82_b9b1_d137ab6eb324.slice/crio-419cc2de6ae4503cfbe7283b2579369ebbad7c9ee0116b94b79c94a06db6406a WatchSource:0}: Error finding container 419cc2de6ae4503cfbe7283b2579369ebbad7c9ee0116b94b79c94a06db6406a: Status 404 returned error can't find the container with id 419cc2de6ae4503cfbe7283b2579369ebbad7c9ee0116b94b79c94a06db6406a
Jul 27 17:30:06.428286 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:06.428247    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c" event={"ID":"04f8702c-7f90-4b82-b9b1-d137ab6eb324","Type":"ContainerStarted","Data":"419cc2de6ae4503cfbe7283b2579369ebbad7c9ee0116b94b79c94a06db6406a"}
Jul 27 17:30:09.018651 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:09.018613    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:09.121924 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:09.121835    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:09.446297 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:09.446210    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c" event={"ID":"04f8702c-7f90-4b82-b9b1-d137ab6eb324","Type":"ContainerStarted","Data":"e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1"}
Jul 27 17:30:09.446451 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:09.446339    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:09.465785 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:09.465732    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c" podStartSLOduration=1.873625184 podStartE2EDuration="4.465715046s" totalImagesPullingTime="2.592089862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:30:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:30:09.464098886 +0000 UTC m=+547.172247204" watchObservedRunningTime="2026-07-27 17:30:09.465715046 +0000 UTC m=+547.173863360"
Jul 27 17:30:11.856070 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:11.855992    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:20.453940 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:20.453906    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:21.495468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:21.495424    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-blr6c"]
Jul 27 17:30:21.495941 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:21.495642    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c" podUID="04f8702c-7f90-4b82-b9b1-d137ab6eb324" containerName="manager" containerID="cri-o://e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1" gracePeriod=30
Jul 27 17:30:21.749073 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:21.749009    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:21.884288 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:21.884255    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04f8702c-7f90-4b82-b9b1-d137ab6eb324-webhook-cert\") pod \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\" (UID: \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\") "
Jul 27 17:30:21.884502 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:21.884318    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwnpr\" (UniqueName: \"kubernetes.io/projected/04f8702c-7f90-4b82-b9b1-d137ab6eb324-kube-api-access-fwnpr\") pod \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\" (UID: \"04f8702c-7f90-4b82-b9b1-d137ab6eb324\") "
Jul 27 17:30:21.886718 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:21.886669    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04f8702c-7f90-4b82-b9b1-d137ab6eb324-kube-api-access-fwnpr" (OuterVolumeSpecName: "kube-api-access-fwnpr") pod "04f8702c-7f90-4b82-b9b1-d137ab6eb324" (UID: "04f8702c-7f90-4b82-b9b1-d137ab6eb324"). InnerVolumeSpecName "kube-api-access-fwnpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:30:21.886718 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:21.886669    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04f8702c-7f90-4b82-b9b1-d137ab6eb324-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "04f8702c-7f90-4b82-b9b1-d137ab6eb324" (UID: "04f8702c-7f90-4b82-b9b1-d137ab6eb324"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:30:21.985229 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:21.985189    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04f8702c-7f90-4b82-b9b1-d137ab6eb324-webhook-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:30:21.985229 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:21.985220    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwnpr\" (UniqueName: \"kubernetes.io/projected/04f8702c-7f90-4b82-b9b1-d137ab6eb324-kube-api-access-fwnpr\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:30:22.509508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:22.509468    2584 generic.go:356] "Generic (PLEG): container finished" podID="04f8702c-7f90-4b82-b9b1-d137ab6eb324" containerID="e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1" exitCode=0
Jul 27 17:30:22.509508 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:22.509516    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c"
Jul 27 17:30:22.510097 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:22.509528    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c" event={"ID":"04f8702c-7f90-4b82-b9b1-d137ab6eb324","Type":"ContainerDied","Data":"e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1"}
Jul 27 17:30:22.510097 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:22.509584    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b4cbfdf4-blr6c" event={"ID":"04f8702c-7f90-4b82-b9b1-d137ab6eb324","Type":"ContainerDied","Data":"419cc2de6ae4503cfbe7283b2579369ebbad7c9ee0116b94b79c94a06db6406a"}
Jul 27 17:30:22.510097 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:22.509618    2584 scope.go:117] "RemoveContainer" containerID="e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1"
Jul 27 17:30:22.518978 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:22.518958    2584 scope.go:117] "RemoveContainer" containerID="e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1"
Jul 27 17:30:22.519254 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:30:22.519234    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1\": container with ID starting with e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1 not found: ID does not exist" containerID="e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1"
Jul 27 17:30:22.519328 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:22.519262    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1"} err="failed to get container status \"e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1\": rpc error: code = NotFound desc = could not find container \"e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1\": container with ID starting with e1b86e9b3ece0901785fd24e19349c8a07c06e5fd04b29eee3f3232a88193dd1 not found: ID does not exist"
Jul 27 17:30:22.532718 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:22.532664    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-blr6c"]
Jul 27 17:30:22.536187 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:22.536162    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57b4cbfdf4-blr6c"]
Jul 27 17:30:22.822987 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:22.822954    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04f8702c-7f90-4b82-b9b1-d137ab6eb324" path="/var/lib/kubelet/pods/04f8702c-7f90-4b82-b9b1-d137ab6eb324/volumes"
Jul 27 17:30:41.184360 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:41.184324    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:42.339276 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.339243    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"]
Jul 27 17:30:42.339709 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.339680    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04f8702c-7f90-4b82-b9b1-d137ab6eb324" containerName="manager"
Jul 27 17:30:42.339709 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.339711    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="04f8702c-7f90-4b82-b9b1-d137ab6eb324" containerName="manager"
Jul 27 17:30:42.339798 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.339761    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="04f8702c-7f90-4b82-b9b1-d137ab6eb324" containerName="manager"
Jul 27 17:30:42.347341 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.347323    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.352047 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.352021    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 17:30:42.352047 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.352046    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 27 17:30:42.352315 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.352052    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qxgf2\""
Jul 27 17:30:42.352315 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.352171    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 27 17:30:42.376878 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.376848    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"]
Jul 27 17:30:42.466021 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.465989    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed47e6fe-a625-4d4e-9148-c5bfee594724-maas-api-tls\") pod \"maas-api-5d8cd8686-ft867\" (UID: \"ed47e6fe-a625-4d4e-9148-c5bfee594724\") " pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.466021 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.466021    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed47e6fe-a625-4d4e-9148-c5bfee594724-openshift-service-ca\") pod \"maas-api-5d8cd8686-ft867\" (UID: \"ed47e6fe-a625-4d4e-9148-c5bfee594724\") " pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.466232 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.466082    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g244r\" (UniqueName: \"kubernetes.io/projected/ed47e6fe-a625-4d4e-9148-c5bfee594724-kube-api-access-g244r\") pod \"maas-api-5d8cd8686-ft867\" (UID: \"ed47e6fe-a625-4d4e-9148-c5bfee594724\") " pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.567237 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.567205    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g244r\" (UniqueName: \"kubernetes.io/projected/ed47e6fe-a625-4d4e-9148-c5bfee594724-kube-api-access-g244r\") pod \"maas-api-5d8cd8686-ft867\" (UID: \"ed47e6fe-a625-4d4e-9148-c5bfee594724\") " pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.567408 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.567295    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed47e6fe-a625-4d4e-9148-c5bfee594724-maas-api-tls\") pod \"maas-api-5d8cd8686-ft867\" (UID: \"ed47e6fe-a625-4d4e-9148-c5bfee594724\") " pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.567408 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.567312    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed47e6fe-a625-4d4e-9148-c5bfee594724-openshift-service-ca\") pod \"maas-api-5d8cd8686-ft867\" (UID: \"ed47e6fe-a625-4d4e-9148-c5bfee594724\") " pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.567900 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.567880    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed47e6fe-a625-4d4e-9148-c5bfee594724-openshift-service-ca\") pod \"maas-api-5d8cd8686-ft867\" (UID: \"ed47e6fe-a625-4d4e-9148-c5bfee594724\") " pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.569871 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.569845    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed47e6fe-a625-4d4e-9148-c5bfee594724-maas-api-tls\") pod \"maas-api-5d8cd8686-ft867\" (UID: \"ed47e6fe-a625-4d4e-9148-c5bfee594724\") " pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.576670 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.576645    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g244r\" (UniqueName: \"kubernetes.io/projected/ed47e6fe-a625-4d4e-9148-c5bfee594724-kube-api-access-g244r\") pod \"maas-api-5d8cd8686-ft867\" (UID: \"ed47e6fe-a625-4d4e-9148-c5bfee594724\") " pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.658037 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.657951    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:42.804605 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.804568    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"]
Jul 27 17:30:42.922897 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.922811    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"]
Jul 27 17:30:42.928069 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.928049    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"
Jul 27 17:30:42.931089 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.931066    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 27 17:30:42.931254 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.931236    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tf4c5\""
Jul 27 17:30:42.936758 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.936734    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"]
Jul 27 17:30:42.970581 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.970548    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6p2f\" (UniqueName: \"kubernetes.io/projected/0f06b985-67d7-49e9-8a8f-a9e2c62f7367-kube-api-access-p6p2f\") pod \"payload-pre-processing-687dbb6cf6-kg5kf\" (UID: \"0f06b985-67d7-49e9-8a8f-a9e2c62f7367\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"
Jul 27 17:30:42.970791 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:42.970725    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f06b985-67d7-49e9-8a8f-a9e2c62f7367-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-kg5kf\" (UID: \"0f06b985-67d7-49e9-8a8f-a9e2c62f7367\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"
Jul 27 17:30:43.071544 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:43.071502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6p2f\" (UniqueName: \"kubernetes.io/projected/0f06b985-67d7-49e9-8a8f-a9e2c62f7367-kube-api-access-p6p2f\") pod \"payload-pre-processing-687dbb6cf6-kg5kf\" (UID: \"0f06b985-67d7-49e9-8a8f-a9e2c62f7367\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"
Jul 27 17:30:43.071780 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:43.071625    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f06b985-67d7-49e9-8a8f-a9e2c62f7367-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-kg5kf\" (UID: \"0f06b985-67d7-49e9-8a8f-a9e2c62f7367\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"
Jul 27 17:30:43.072350 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:43.072325    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f06b985-67d7-49e9-8a8f-a9e2c62f7367-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-kg5kf\" (UID: \"0f06b985-67d7-49e9-8a8f-a9e2c62f7367\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"
Jul 27 17:30:43.081607 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:43.081587    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6p2f\" (UniqueName: \"kubernetes.io/projected/0f06b985-67d7-49e9-8a8f-a9e2c62f7367-kube-api-access-p6p2f\") pod \"payload-pre-processing-687dbb6cf6-kg5kf\" (UID: \"0f06b985-67d7-49e9-8a8f-a9e2c62f7367\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"
Jul 27 17:30:43.240832 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:43.240739    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"
Jul 27 17:30:43.380300 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:43.380270    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"]
Jul 27 17:30:43.382245 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:30:43.382209    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f06b985_67d7_49e9_8a8f_a9e2c62f7367.slice/crio-1385e1bdc74bdc3cdb1a1f8e582356202640e2134a8ca463723883958a56ee46 WatchSource:0}: Error finding container 1385e1bdc74bdc3cdb1a1f8e582356202640e2134a8ca463723883958a56ee46: Status 404 returned error can't find the container with id 1385e1bdc74bdc3cdb1a1f8e582356202640e2134a8ca463723883958a56ee46
Jul 27 17:30:43.617641 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:43.617600    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf" event={"ID":"0f06b985-67d7-49e9-8a8f-a9e2c62f7367","Type":"ContainerStarted","Data":"1385e1bdc74bdc3cdb1a1f8e582356202640e2134a8ca463723883958a56ee46"}
Jul 27 17:30:43.618752 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:43.618732    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867" event={"ID":"ed47e6fe-a625-4d4e-9148-c5bfee594724","Type":"ContainerStarted","Data":"d49bf1f30042a4f7d96a8d084eade21405fe8541267180dd56c79005d9299cae"}
Jul 27 17:30:45.362707 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:45.362639    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:45.633278 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:45.633187    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf" event={"ID":"0f06b985-67d7-49e9-8a8f-a9e2c62f7367","Type":"ContainerStarted","Data":"f538e9922038b2d474cc5ad303bb9b8add34f2b481744f3f17a56192a79a3b32"}
Jul 27 17:30:45.633278 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:45.633261    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"
Jul 27 17:30:47.034680 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:47.034641    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:47.645532 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:47.645495    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867" event={"ID":"ed47e6fe-a625-4d4e-9148-c5bfee594724","Type":"ContainerStarted","Data":"eccf057cce0c072c5aaaef9e8f36dd0b3ea8e3dd917e757b2d6eaa33b5ac176d"}
Jul 27 17:30:47.645724 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:47.645554    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:47.666319 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:47.666254    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867" podStartSLOduration=1.355146874 podStartE2EDuration="5.666231938s" totalImagesPullingTime="4.311085064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:30:47.664579559 +0000 UTC m=+585.372727877" watchObservedRunningTime="2026-07-27 17:30:47.666231938 +0000 UTC m=+585.374380267"
Jul 27 17:30:47.667318 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:47.667281    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf" podStartSLOduration=4.031247078 podStartE2EDuration="5.667271526s" totalImagesPullingTime="1.636024448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:30:45.653814813 +0000 UTC m=+583.361963131" watchObservedRunningTime="2026-07-27 17:30:47.667271526 +0000 UTC m=+585.375419847"
Jul 27 17:30:53.656886 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:53.656854    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5d8cd8686-ft867"
Jul 27 17:30:54.669143 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:54.669109    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:54.777135 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:54.777103    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:56.641852 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:56.641820    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-kg5kf"
Jul 27 17:30:59.584165 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.584124    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"]
Jul 27 17:30:59.587633 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.587611    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.590535 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.590509    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zlcs9\""
Jul 27 17:30:59.590650 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.590590    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 17:30:59.591893 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.591876    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 17:30:59.591996 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.591892    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 27 17:30:59.598336 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.598313    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"]
Jul 27 17:30:59.614836 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.614792    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:30:59.738904 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.738864    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5f6x\" (UniqueName: \"kubernetes.io/projected/30ea5c73-1869-42e4-898b-2226f7d9ad5d-kube-api-access-l5f6x\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.739476 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.739446    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ea5c73-1869-42e4-898b-2226f7d9ad5d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.739615 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.739590    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.739679 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.739629    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.739679 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.739656    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.739821 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.739777    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.841138 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.841033    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.841138 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.841075    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.841138 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.841098    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.841138 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.841136    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.841473 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.841179    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5f6x\" (UniqueName: \"kubernetes.io/projected/30ea5c73-1869-42e4-898b-2226f7d9ad5d-kube-api-access-l5f6x\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.841473 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.841263    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ea5c73-1869-42e4-898b-2226f7d9ad5d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.841560 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.841521    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.841672 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.841647    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.841753 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.841660    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.843613 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.843594    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ea5c73-1869-42e4-898b-2226f7d9ad5d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.843915 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.843897    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ea5c73-1869-42e4-898b-2226f7d9ad5d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.850076 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.850043    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5f6x\" (UniqueName: \"kubernetes.io/projected/30ea5c73-1869-42e4-898b-2226f7d9ad5d-kube-api-access-l5f6x\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q\" (UID: \"30ea5c73-1869-42e4-898b-2226f7d9ad5d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:30:59.900132 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:30:59.900095    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:31:00.050964 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:00.050942    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"]
Jul 27 17:31:00.053008 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:31:00.052982    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ea5c73_1869_42e4_898b_2226f7d9ad5d.slice/crio-f666f9095b075b767cbc82ba90e8bb453f29c09b4325299be0ca49de72c0ac63 WatchSource:0}: Error finding container f666f9095b075b767cbc82ba90e8bb453f29c09b4325299be0ca49de72c0ac63: Status 404 returned error can't find the container with id f666f9095b075b767cbc82ba90e8bb453f29c09b4325299be0ca49de72c0ac63
Jul 27 17:31:00.716174 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:00.716135    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q" event={"ID":"30ea5c73-1869-42e4-898b-2226f7d9ad5d","Type":"ContainerStarted","Data":"f666f9095b075b767cbc82ba90e8bb453f29c09b4325299be0ca49de72c0ac63"}
Jul 27 17:31:01.722954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:01.722915    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q" event={"ID":"30ea5c73-1869-42e4-898b-2226f7d9ad5d","Type":"ContainerStarted","Data":"b55d6d762cd13bc6785ffbcf2d7f6a6228e91924727a853cb6f78f8da446638b"}
Jul 27 17:31:01.744109 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:01.744036    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q" podStartSLOduration=1.737870332 podStartE2EDuration="2.744018519s" totalImagesPullingTime="1.006148187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:30:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:31:01.742386827 +0000 UTC m=+599.450535156" watchObservedRunningTime="2026-07-27 17:31:01.744018519 +0000 UTC m=+599.452166837"
Jul 27 17:31:06.262940 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.262842    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"]
Jul 27 17:31:06.266574 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.266557    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.269536 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.269515    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 27 17:31:06.276657 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.276631    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"]
Jul 27 17:31:06.287636 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.287614    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:31:06.407427 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.407381    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1777c1b-130d-48a9-8739-8d75b26fd1aa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.407644 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.407441    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.407644 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.407529    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.407644 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.407611    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm764\" (UniqueName: \"kubernetes.io/projected/f1777c1b-130d-48a9-8739-8d75b26fd1aa-kube-api-access-rm764\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.407644 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.407642    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.407860 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.407772    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.508317 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.508283    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm764\" (UniqueName: \"kubernetes.io/projected/f1777c1b-130d-48a9-8739-8d75b26fd1aa-kube-api-access-rm764\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.508317 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.508325    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.508584 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.508369    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.508584 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.508493    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1777c1b-130d-48a9-8739-8d75b26fd1aa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.508584 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.508552    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.508800 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.508585    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.508800 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.508791    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.508902 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.508867    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.509000 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.508976    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.510968 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.510943    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1777c1b-130d-48a9-8739-8d75b26fd1aa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.511162 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.511141    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1777c1b-130d-48a9-8739-8d75b26fd1aa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.518604 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.518540    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm764\" (UniqueName: \"kubernetes.io/projected/f1777c1b-130d-48a9-8739-8d75b26fd1aa-kube-api-access-rm764\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rd9f\" (UID: \"f1777c1b-130d-48a9-8739-8d75b26fd1aa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.578101 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.578067    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:06.719679 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.719652    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"]
Jul 27 17:31:06.723030 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:31:06.722997    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1777c1b_130d_48a9_8739_8d75b26fd1aa.slice/crio-3089b0330f66d49fb786270125c2df6608ea6a7bbb09e63aca24fad1eaaad73a WatchSource:0}: Error finding container 3089b0330f66d49fb786270125c2df6608ea6a7bbb09e63aca24fad1eaaad73a: Status 404 returned error can't find the container with id 3089b0330f66d49fb786270125c2df6608ea6a7bbb09e63aca24fad1eaaad73a
Jul 27 17:31:06.750010 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:06.749978    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f" event={"ID":"f1777c1b-130d-48a9-8739-8d75b26fd1aa","Type":"ContainerStarted","Data":"3089b0330f66d49fb786270125c2df6608ea6a7bbb09e63aca24fad1eaaad73a"}
Jul 27 17:31:07.757169 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:07.757127    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f" event={"ID":"f1777c1b-130d-48a9-8739-8d75b26fd1aa","Type":"ContainerStarted","Data":"58fdcf8a3180d8d0366338bf9cec91690b6dd4fb2f5f12c83da5bdaab0dcb10b"}
Jul 27 17:31:07.778096 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:07.778033    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f" podStartSLOduration=1.604231495 podStartE2EDuration="1.778013015s" totalImagesPullingTime="173.78152ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:31:07.775854731 +0000 UTC m=+605.484003051" watchObservedRunningTime="2026-07-27 17:31:07.778013015 +0000 UTC m=+605.486161333"
Jul 27 17:31:09.900674 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:09.900630    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:31:09.901098 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:09.900775    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:31:09.913436 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:09.913408    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:31:10.784194 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:10.784167    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q"
Jul 27 17:31:12.696114 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:12.696079    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:31:16.579175 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:16.579139    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:16.579552 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:16.579205    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:16.592100 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:16.592066    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:16.813460 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:16.813429    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rd9f"
Jul 27 17:31:34.407206 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:34.407122    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:31:43.376760 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.376721    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"]
Jul 27 17:31:43.380962 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.380941    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.385328 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.385306    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 27 17:31:43.388755 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.388729    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.388858 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.388798    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.388858 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.388824    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.388976 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.388857    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.388976 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.388918    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6acf61-21bd-4b12-b217-7beec8c092ca-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.388976 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.388961    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9s6f\" (UniqueName: \"kubernetes.io/projected/5e6acf61-21bd-4b12-b217-7beec8c092ca-kube-api-access-n9s6f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.403094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.403060    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"]
Jul 27 17:31:43.438236 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.438174    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:31:43.489568 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.489515    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9s6f\" (UniqueName: \"kubernetes.io/projected/5e6acf61-21bd-4b12-b217-7beec8c092ca-kube-api-access-n9s6f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.489820 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.489585    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.489820 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.489627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.489820 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.489646    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.489820 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.489668    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.489820 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.489745    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6acf61-21bd-4b12-b217-7beec8c092ca-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.490209 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.490187    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.490283 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.490207    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.490283 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.490272    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.492170 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.492146    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6acf61-21bd-4b12-b217-7beec8c092ca-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.492541 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.492521    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6acf61-21bd-4b12-b217-7beec8c092ca-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.499375 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.499349    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9s6f\" (UniqueName: \"kubernetes.io/projected/5e6acf61-21bd-4b12-b217-7beec8c092ca-kube-api-access-n9s6f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vqnpc\" (UID: \"5e6acf61-21bd-4b12-b217-7beec8c092ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.691959 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.691848    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:43.838142 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.838118    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"]
Jul 27 17:31:43.840595 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:31:43.840564    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6acf61_21bd_4b12_b217_7beec8c092ca.slice/crio-75aaef7a68800adb1e90fcd5f5b5dca1c01e9d3ae572723bb8c3365e80b2180c WatchSource:0}: Error finding container 75aaef7a68800adb1e90fcd5f5b5dca1c01e9d3ae572723bb8c3365e80b2180c: Status 404 returned error can't find the container with id 75aaef7a68800adb1e90fcd5f5b5dca1c01e9d3ae572723bb8c3365e80b2180c
Jul 27 17:31:43.842051 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.842029    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:31:43.932514 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:43.932479    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc" event={"ID":"5e6acf61-21bd-4b12-b217-7beec8c092ca","Type":"ContainerStarted","Data":"75aaef7a68800adb1e90fcd5f5b5dca1c01e9d3ae572723bb8c3365e80b2180c"}
Jul 27 17:31:44.939524 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:44.939484    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc" event={"ID":"5e6acf61-21bd-4b12-b217-7beec8c092ca","Type":"ContainerStarted","Data":"cbb1ded7166f71ee9e937ce26a903bac366a770c9f32b5f8677e8ab5f46edc9e"}
Jul 27 17:31:44.961005 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:44.960949    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc" podStartSLOduration=1.7762992290000001 podStartE2EDuration="1.960927469s" totalImagesPullingTime="184.62824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:31:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:31:44.959032867 +0000 UTC m=+642.667181200" watchObservedRunningTime="2026-07-27 17:31:44.960927469 +0000 UTC m=+642.669075786"
Jul 27 17:31:53.692767 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:53.692719    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:53.693172 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:53.692805    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:53.705395 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:53.705370    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:31:54.002159 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:31:54.002066    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vqnpc"
Jul 27 17:32:16.913272 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:32:16.913228    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:18.911829 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:33:18.911798    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:19.004108 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:33:19.004073    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:20.020655 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:33:20.020622    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:20.141273 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:33:20.141224    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:07.494715 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.494640    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7975dbccc5-jjm65"]
Jul 27 17:35:07.498245 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.498220    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:07.502449 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.502424    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 17:35:07.502449 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.502446    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bps2r\""
Jul 27 17:35:07.502650 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.502477    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 17:35:07.508761 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.508733    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7975dbccc5-jjm65"]
Jul 27 17:35:07.626170 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.626134    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gln4s\" (UniqueName: \"kubernetes.io/projected/ede3506c-f903-4715-93de-0aa97795ec77-kube-api-access-gln4s\") pod \"maas-controller-7975dbccc5-jjm65\" (UID: \"ede3506c-f903-4715-93de-0aa97795ec77\") " pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:07.626364 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.626269    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ede3506c-f903-4715-93de-0aa97795ec77-webhook-cert\") pod \"maas-controller-7975dbccc5-jjm65\" (UID: \"ede3506c-f903-4715-93de-0aa97795ec77\") " pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:07.727837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.727792    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gln4s\" (UniqueName: \"kubernetes.io/projected/ede3506c-f903-4715-93de-0aa97795ec77-kube-api-access-gln4s\") pod \"maas-controller-7975dbccc5-jjm65\" (UID: \"ede3506c-f903-4715-93de-0aa97795ec77\") " pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:07.728042 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.727886    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ede3506c-f903-4715-93de-0aa97795ec77-webhook-cert\") pod \"maas-controller-7975dbccc5-jjm65\" (UID: \"ede3506c-f903-4715-93de-0aa97795ec77\") " pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:07.730448 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.730423    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ede3506c-f903-4715-93de-0aa97795ec77-webhook-cert\") pod \"maas-controller-7975dbccc5-jjm65\" (UID: \"ede3506c-f903-4715-93de-0aa97795ec77\") " pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:07.737499 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.737470    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gln4s\" (UniqueName: \"kubernetes.io/projected/ede3506c-f903-4715-93de-0aa97795ec77-kube-api-access-gln4s\") pod \"maas-controller-7975dbccc5-jjm65\" (UID: \"ede3506c-f903-4715-93de-0aa97795ec77\") " pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:07.810439 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.810392    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:07.947433 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.947392    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:07.954090 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:07.954047    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7975dbccc5-jjm65"]
Jul 27 17:35:07.956704 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:35:07.956664    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podede3506c_f903_4715_93de_0aa97795ec77.slice/crio-bffda0b83eb62325ca39a018d1b1fd0274a52af65d5d2b80c21df7c5818f83c6 WatchSource:0}: Error finding container bffda0b83eb62325ca39a018d1b1fd0274a52af65d5d2b80c21df7c5818f83c6: Status 404 returned error can't find the container with id bffda0b83eb62325ca39a018d1b1fd0274a52af65d5d2b80c21df7c5818f83c6
Jul 27 17:35:08.028051 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:08.028017    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:08.965925 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:08.965893    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7975dbccc5-jjm65" event={"ID":"ede3506c-f903-4715-93de-0aa97795ec77","Type":"ContainerStarted","Data":"97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac"}
Jul 27 17:35:08.966334 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:08.965943    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7975dbccc5-jjm65" event={"ID":"ede3506c-f903-4715-93de-0aa97795ec77","Type":"ContainerStarted","Data":"bffda0b83eb62325ca39a018d1b1fd0274a52af65d5d2b80c21df7c5818f83c6"}
Jul 27 17:35:08.966334 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:08.966007    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:08.985441 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:08.985386    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7975dbccc5-jjm65" podStartSLOduration=1.496243365 podStartE2EDuration="1.985369222s" totalImagesPullingTime="489.125857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:35:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:35:08.984359498 +0000 UTC m=+846.692507818" watchObservedRunningTime="2026-07-27 17:35:08.985369222 +0000 UTC m=+846.693517540"
Jul 27 17:35:19.973342 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:19.973310    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:24.503445 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:24.503407    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:24.598920 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:24.598885    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7975dbccc5-jjm65"]
Jul 27 17:35:24.599119 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:24.599098    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7975dbccc5-jjm65" podUID="ede3506c-f903-4715-93de-0aa97795ec77" containerName="manager" containerID="cri-o://97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac" gracePeriod=30
Jul 27 17:35:24.853884 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:24.853859    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:24.905642 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:24.905611    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ede3506c-f903-4715-93de-0aa97795ec77-webhook-cert\") pod \"ede3506c-f903-4715-93de-0aa97795ec77\" (UID: \"ede3506c-f903-4715-93de-0aa97795ec77\") "
Jul 27 17:35:24.905856 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:24.905744    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gln4s\" (UniqueName: \"kubernetes.io/projected/ede3506c-f903-4715-93de-0aa97795ec77-kube-api-access-gln4s\") pod \"ede3506c-f903-4715-93de-0aa97795ec77\" (UID: \"ede3506c-f903-4715-93de-0aa97795ec77\") "
Jul 27 17:35:24.907897 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:24.907870    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ede3506c-f903-4715-93de-0aa97795ec77-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ede3506c-f903-4715-93de-0aa97795ec77" (UID: "ede3506c-f903-4715-93de-0aa97795ec77"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:35:24.908022 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:24.907979    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ede3506c-f903-4715-93de-0aa97795ec77-kube-api-access-gln4s" (OuterVolumeSpecName: "kube-api-access-gln4s") pod "ede3506c-f903-4715-93de-0aa97795ec77" (UID: "ede3506c-f903-4715-93de-0aa97795ec77"). InnerVolumeSpecName "kube-api-access-gln4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:35:25.006412 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.006377    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ede3506c-f903-4715-93de-0aa97795ec77-webhook-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:35:25.006412 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.006406    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gln4s\" (UniqueName: \"kubernetes.io/projected/ede3506c-f903-4715-93de-0aa97795ec77-kube-api-access-gln4s\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:35:25.046532 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.046446    2584 generic.go:356] "Generic (PLEG): container finished" podID="ede3506c-f903-4715-93de-0aa97795ec77" containerID="97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac" exitCode=0
Jul 27 17:35:25.046532 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.046517    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7975dbccc5-jjm65"
Jul 27 17:35:25.046743 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.046518    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7975dbccc5-jjm65" event={"ID":"ede3506c-f903-4715-93de-0aa97795ec77","Type":"ContainerDied","Data":"97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac"}
Jul 27 17:35:25.046743 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.046668    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7975dbccc5-jjm65" event={"ID":"ede3506c-f903-4715-93de-0aa97795ec77","Type":"ContainerDied","Data":"bffda0b83eb62325ca39a018d1b1fd0274a52af65d5d2b80c21df7c5818f83c6"}
Jul 27 17:35:25.046743 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.046714    2584 scope.go:117] "RemoveContainer" containerID="97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac"
Jul 27 17:35:25.056407 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.056387    2584 scope.go:117] "RemoveContainer" containerID="97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac"
Jul 27 17:35:25.056678 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:35:25.056656    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac\": container with ID starting with 97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac not found: ID does not exist" containerID="97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac"
Jul 27 17:35:25.056797 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.056703    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac"} err="failed to get container status \"97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac\": rpc error: code = NotFound desc = could not find container \"97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac\": container with ID starting with 97b7cbd9bf3b0730e659987613e51cdb319af4b32d6cca35ef683acd1a05d3ac not found: ID does not exist"
Jul 27 17:35:25.068890 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.068864    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7975dbccc5-jjm65"]
Jul 27 17:35:25.072783 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.072743    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7975dbccc5-jjm65"]
Jul 27 17:35:25.725717 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:25.725654    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:26.823081 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:26.823047    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ede3506c-f903-4715-93de-0aa97795ec77" path="/var/lib/kubelet/pods/ede3506c-f903-4715-93de-0aa97795ec77/volumes"
Jul 27 17:35:53.818545 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:53.818446    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:55.485253 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:55.485212    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:59.057104 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:59.057051    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:59.158839 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:35:59.158798    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:07.528031 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:07.527997    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:07.629744 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:07.629703    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:07.758838 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:07.758797    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:13.395285 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:13.395250    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:13.453263 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:13.453222    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:13.553783 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:13.553737    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:13.725647 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:13.725545    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:13.822141 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:13.822075    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:19.564185 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:19.564142    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:19.664597 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:19.664560    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:26.289907 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:26.289876    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:49.373402 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:36:49.373350    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:24.963921 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:42:24.963879    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:43:38.021980 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:43:38.021901    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:43:38.123643 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:43:38.123607    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:00.775064 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:44:00.775013    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:10.486835 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:44:10.486797    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:17.079855 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:44:17.079816    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:17.149173 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:44:17.149141    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:28.759337 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:44:28.759305    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:29.895013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:44:29.894983    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:31.985339 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:44:31.985303    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:35.466850 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:47:35.466796    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:55.951027 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:47:55.950944    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:58.888703 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:47:58.888652    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:58.983009 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:47:58.982975    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:59.333347 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:47:59.333318    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:05.073436 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:05.073402    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:05.151270 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:05.151242    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:05.303209 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:05.303178    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:05.590170 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:05.590127    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:06.045354 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:06.045274    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:08.446280 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.446245    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:08.520449 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.520409    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"]
Jul 27 17:48:08.520976 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.520957    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ede3506c-f903-4715-93de-0aa97795ec77" containerName="manager"
Jul 27 17:48:08.521046 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.520978    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ede3506c-f903-4715-93de-0aa97795ec77" containerName="manager"
Jul 27 17:48:08.521099 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.521061    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ede3506c-f903-4715-93de-0aa97795ec77" containerName="manager"
Jul 27 17:48:08.524197 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.524180    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:08.527094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.527060    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5bebd9b0-serving-cert\""
Jul 27 17:48:08.534460 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.534435    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"]
Jul 27 17:48:08.540107 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.540077    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:08.610480 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.610439    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf320069-b358-4233-a7ef-1a11df5697bf-openshift-service-ca\") pod \"maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:08.610741 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.610593    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49gpr\" (UniqueName: \"kubernetes.io/projected/cf320069-b358-4233-a7ef-1a11df5697bf-kube-api-access-49gpr\") pod \"maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:08.610741 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.610633    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf320069-b358-4233-a7ef-1a11df5697bf-maas-api-tls\") pod \"maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:08.627822 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.627780    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:08.711172 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.711080    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49gpr\" (UniqueName: \"kubernetes.io/projected/cf320069-b358-4233-a7ef-1a11df5697bf-kube-api-access-49gpr\") pod \"maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:08.711172 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.711123    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf320069-b358-4233-a7ef-1a11df5697bf-maas-api-tls\") pod \"maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:08.711172 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.711156    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf320069-b358-4233-a7ef-1a11df5697bf-openshift-service-ca\") pod \"maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:08.711419 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:48:08.711304    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-serving-cert: secret "maas-api-e2e-ait-5bebd9b0-serving-cert" not found
Jul 27 17:48:08.711419 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:48:08.711371    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf320069-b358-4233-a7ef-1a11df5697bf-maas-api-tls podName:cf320069-b358-4233-a7ef-1a11df5697bf nodeName:}" failed. No retries permitted until 2026-07-27 17:48:09.211355889 +0000 UTC m=+1626.919504185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf320069-b358-4233-a7ef-1a11df5697bf-maas-api-tls") pod "maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq" (UID: "cf320069-b358-4233-a7ef-1a11df5697bf") : secret "maas-api-e2e-ait-5bebd9b0-serving-cert" not found
Jul 27 17:48:08.711904 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.711885    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf320069-b358-4233-a7ef-1a11df5697bf-openshift-service-ca\") pod \"maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:08.720357 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:08.720330    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49gpr\" (UniqueName: \"kubernetes.io/projected/cf320069-b358-4233-a7ef-1a11df5697bf-kube-api-access-49gpr\") pod \"maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:09.049935 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.049836    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:09.214995 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.214963    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf320069-b358-4233-a7ef-1a11df5697bf-maas-api-tls\") pod \"maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:09.217645 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.217615    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf320069-b358-4233-a7ef-1a11df5697bf-maas-api-tls\") pod \"maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:09.239708 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.239638    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:09.342567 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.342531    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:09.380420 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.380380    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"]
Jul 27 17:48:09.384081 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.384064    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:09.387185 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.387159    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5bebd9b0\""
Jul 27 17:48:09.387334 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.387170    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5bebd9b0-dockercfg-6cpjq\""
Jul 27 17:48:09.393786 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.393758    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"]
Jul 27 17:48:09.417270 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.417234    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtlxn\" (UniqueName: \"kubernetes.io/projected/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-kube-api-access-rtlxn\") pod \"payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd\" (UID: \"a3746261-9cf6-4dd8-a78b-b11afc6e44b9\") " pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:09.417449 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.417354    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-plugins-config-volume\") pod \"payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd\" (UID: \"a3746261-9cf6-4dd8-a78b-b11afc6e44b9\") " pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:09.437096 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.437057    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:09.517962 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.517931    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtlxn\" (UniqueName: \"kubernetes.io/projected/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-kube-api-access-rtlxn\") pod \"payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd\" (UID: \"a3746261-9cf6-4dd8-a78b-b11afc6e44b9\") " pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:09.518294 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.518008    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-plugins-config-volume\") pod \"payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd\" (UID: \"a3746261-9cf6-4dd8-a78b-b11afc6e44b9\") " pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:09.518737 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.518679    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-plugins-config-volume\") pod \"payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd\" (UID: \"a3746261-9cf6-4dd8-a78b-b11afc6e44b9\") " pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:09.526539 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.526510    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtlxn\" (UniqueName: \"kubernetes.io/projected/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-kube-api-access-rtlxn\") pod \"payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd\" (UID: \"a3746261-9cf6-4dd8-a78b-b11afc6e44b9\") " pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:09.590504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.590474    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"]
Jul 27 17:48:09.593585 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:48:09.593508    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf320069_b358_4233_a7ef_1a11df5697bf.slice/crio-5cb278be72219477737d5cd0d47d9b7e85f99a2ef6389f87413a95691f7f6fab WatchSource:0}: Error finding container 5cb278be72219477737d5cd0d47d9b7e85f99a2ef6389f87413a95691f7f6fab: Status 404 returned error can't find the container with id 5cb278be72219477737d5cd0d47d9b7e85f99a2ef6389f87413a95691f7f6fab
Jul 27 17:48:09.594929 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.594911    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:48:09.697108 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.697069    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:09.839605 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.839571    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"]
Jul 27 17:48:09.842957 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:48:09.842922    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3746261_9cf6_4dd8_a78b_b11afc6e44b9.slice/crio-8a376c4fe73da59419730ce7427f93bd626b6cf6dac2402b69b6936a3ba0d806 WatchSource:0}: Error finding container 8a376c4fe73da59419730ce7427f93bd626b6cf6dac2402b69b6936a3ba0d806: Status 404 returned error can't find the container with id 8a376c4fe73da59419730ce7427f93bd626b6cf6dac2402b69b6936a3ba0d806
Jul 27 17:48:09.850877 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.850842    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq" event={"ID":"cf320069-b358-4233-a7ef-1a11df5697bf","Type":"ContainerStarted","Data":"5cb278be72219477737d5cd0d47d9b7e85f99a2ef6389f87413a95691f7f6fab"}
Jul 27 17:48:09.852059 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:09.852036    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd" event={"ID":"a3746261-9cf6-4dd8-a78b-b11afc6e44b9","Type":"ContainerStarted","Data":"8a376c4fe73da59419730ce7427f93bd626b6cf6dac2402b69b6936a3ba0d806"}
Jul 27 17:48:10.858418 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:10.858381    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq" event={"ID":"cf320069-b358-4233-a7ef-1a11df5697bf","Type":"ContainerStarted","Data":"fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012"}
Jul 27 17:48:10.858908 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:10.858548    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:10.860000 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:10.859971    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd" event={"ID":"a3746261-9cf6-4dd8-a78b-b11afc6e44b9","Type":"ContainerStarted","Data":"218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931"}
Jul 27 17:48:10.860128 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:10.860114    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:10.878782 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:10.878736    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq" podStartSLOduration=2.285473439 podStartE2EDuration="2.878720651s" totalImagesPullingTime="593.247212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:48:10.877046717 +0000 UTC m=+1628.585195037" watchObservedRunningTime="2026-07-27 17:48:10.878720651 +0000 UTC m=+1628.586868968"
Jul 27 17:48:10.895458 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:10.895409    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd" podStartSLOduration=1.8953947 podStartE2EDuration="1.8953947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:48:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:48:10.894481541 +0000 UTC m=+1628.602629856" watchObservedRunningTime="2026-07-27 17:48:10.8953947 +0000 UTC m=+1628.603543018"
Jul 27 17:48:16.871736 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:16.871706    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:21.867681 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:21.867651    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:43.078653 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:43.078607    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:44.400250 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.400207    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"]
Jul 27 17:48:44.400747 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.400425    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq" podUID="cf320069-b358-4233-a7ef-1a11df5697bf" containerName="maas-api" containerID="cri-o://fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012" gracePeriod=30
Jul 27 17:48:44.433627 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.433590    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:44.472595 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.472560    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"]
Jul 27 17:48:44.472825 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.472802    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd" podUID="a3746261-9cf6-4dd8-a78b-b11afc6e44b9" containerName="payload-processing" containerID="cri-o://218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931" gracePeriod=30
Jul 27 17:48:44.512031 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.511983    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:44.608589 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.608559    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:44.734208 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.734179    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:44.766960 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.766928    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:44.789984 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.789950    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf320069-b358-4233-a7ef-1a11df5697bf-openshift-service-ca\") pod \"cf320069-b358-4233-a7ef-1a11df5697bf\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") "
Jul 27 17:48:44.789984 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.789984    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtlxn\" (UniqueName: \"kubernetes.io/projected/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-kube-api-access-rtlxn\") pod \"a3746261-9cf6-4dd8-a78b-b11afc6e44b9\" (UID: \"a3746261-9cf6-4dd8-a78b-b11afc6e44b9\") "
Jul 27 17:48:44.790271 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.790008    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-plugins-config-volume\") pod \"a3746261-9cf6-4dd8-a78b-b11afc6e44b9\" (UID: \"a3746261-9cf6-4dd8-a78b-b11afc6e44b9\") "
Jul 27 17:48:44.790271 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.790043    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49gpr\" (UniqueName: \"kubernetes.io/projected/cf320069-b358-4233-a7ef-1a11df5697bf-kube-api-access-49gpr\") pod \"cf320069-b358-4233-a7ef-1a11df5697bf\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") "
Jul 27 17:48:44.790271 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.790082    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf320069-b358-4233-a7ef-1a11df5697bf-maas-api-tls\") pod \"cf320069-b358-4233-a7ef-1a11df5697bf\" (UID: \"cf320069-b358-4233-a7ef-1a11df5697bf\") "
Jul 27 17:48:44.790442 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.790410    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf320069-b358-4233-a7ef-1a11df5697bf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf320069-b358-4233-a7ef-1a11df5697bf" (UID: "cf320069-b358-4233-a7ef-1a11df5697bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:48:44.790509 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.790430    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3746261-9cf6-4dd8-a78b-b11afc6e44b9" (UID: "a3746261-9cf6-4dd8-a78b-b11afc6e44b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:48:44.792524 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.792494    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf320069-b358-4233-a7ef-1a11df5697bf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf320069-b358-4233-a7ef-1a11df5697bf" (UID: "cf320069-b358-4233-a7ef-1a11df5697bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:48:44.792524 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.792517    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf320069-b358-4233-a7ef-1a11df5697bf-kube-api-access-49gpr" (OuterVolumeSpecName: "kube-api-access-49gpr") pod "cf320069-b358-4233-a7ef-1a11df5697bf" (UID: "cf320069-b358-4233-a7ef-1a11df5697bf"). InnerVolumeSpecName "kube-api-access-49gpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:48:44.792809 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.792791    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-kube-api-access-rtlxn" (OuterVolumeSpecName: "kube-api-access-rtlxn") pod "a3746261-9cf6-4dd8-a78b-b11afc6e44b9" (UID: "a3746261-9cf6-4dd8-a78b-b11afc6e44b9"). InnerVolumeSpecName "kube-api-access-rtlxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:48:44.891180 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.891134    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf320069-b358-4233-a7ef-1a11df5697bf-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:48:44.891180 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.891167    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtlxn\" (UniqueName: \"kubernetes.io/projected/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-kube-api-access-rtlxn\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:48:44.891180 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.891178    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3746261-9cf6-4dd8-a78b-b11afc6e44b9-plugins-config-volume\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:48:44.891180 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.891187    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49gpr\" (UniqueName: \"kubernetes.io/projected/cf320069-b358-4233-a7ef-1a11df5697bf-kube-api-access-49gpr\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:48:44.891180 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:44.891196    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf320069-b358-4233-a7ef-1a11df5697bf-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:48:45.035094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.034991    2584 generic.go:356] "Generic (PLEG): container finished" podID="cf320069-b358-4233-a7ef-1a11df5697bf" containerID="fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012" exitCode=0
Jul 27 17:48:45.035094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.035060    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"
Jul 27 17:48:45.035094 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.035085    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq" event={"ID":"cf320069-b358-4233-a7ef-1a11df5697bf","Type":"ContainerDied","Data":"fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012"}
Jul 27 17:48:45.035377 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.035149    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq" event={"ID":"cf320069-b358-4233-a7ef-1a11df5697bf","Type":"ContainerDied","Data":"5cb278be72219477737d5cd0d47d9b7e85f99a2ef6389f87413a95691f7f6fab"}
Jul 27 17:48:45.035377 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.035175    2584 scope.go:117] "RemoveContainer" containerID="fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012"
Jul 27 17:48:45.036523 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.036497    2584 generic.go:356] "Generic (PLEG): container finished" podID="a3746261-9cf6-4dd8-a78b-b11afc6e44b9" containerID="218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931" exitCode=0
Jul 27 17:48:45.036645 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.036556    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"
Jul 27 17:48:45.036645 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.036555    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd" event={"ID":"a3746261-9cf6-4dd8-a78b-b11afc6e44b9","Type":"ContainerDied","Data":"218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931"}
Jul 27 17:48:45.036645 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.036602    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd" event={"ID":"a3746261-9cf6-4dd8-a78b-b11afc6e44b9","Type":"ContainerDied","Data":"8a376c4fe73da59419730ce7427f93bd626b6cf6dac2402b69b6936a3ba0d806"}
Jul 27 17:48:45.044414 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.044393    2584 scope.go:117] "RemoveContainer" containerID="fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012"
Jul 27 17:48:45.044751 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:48:45.044727    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012\": container with ID starting with fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012 not found: ID does not exist" containerID="fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012"
Jul 27 17:48:45.044829 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.044759    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012"} err="failed to get container status \"fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012\": rpc error: code = NotFound desc = could not find container \"fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012\": container with ID starting with fa5698227e5b9336b34b0bc9f30c4b9f203149f818d7bd9ee695a29ed4cf0012 not found: ID does not exist"
Jul 27 17:48:45.044829 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.044779    2584 scope.go:117] "RemoveContainer" containerID="218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931"
Jul 27 17:48:45.054609 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.054573    2584 scope.go:117] "RemoveContainer" containerID="218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931"
Jul 27 17:48:45.054960 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.054936    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"]
Jul 27 17:48:45.055101 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:48:45.054972    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931\": container with ID starting with 218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931 not found: ID does not exist" containerID="218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931"
Jul 27 17:48:45.055101 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.055005    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931"} err="failed to get container status \"218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931\": rpc error: code = NotFound desc = could not find container \"218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931\": container with ID starting with 218f280fffadf4aaf985f7c6dd8646e9b7ee607d867f9315a42e4eb8416dd931 not found: ID does not exist"
Jul 27 17:48:45.057243 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.057217    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5bebd9b0-5944564d94-h5gbd"]
Jul 27 17:48:45.070971 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.070934    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"]
Jul 27 17:48:45.074968 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:45.074939    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5bebd9b0-d57fc4cf6-6b9qq"]
Jul 27 17:48:46.823358 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:46.823325    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3746261-9cf6-4dd8-a78b-b11afc6e44b9" path="/var/lib/kubelet/pods/a3746261-9cf6-4dd8-a78b-b11afc6e44b9/volumes"
Jul 27 17:48:46.823770 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:46.823744    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf320069-b358-4233-a7ef-1a11df5697bf" path="/var/lib/kubelet/pods/cf320069-b358-4233-a7ef-1a11df5697bf/volumes"
Jul 27 17:48:49.032861 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:49.032829    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:49.183378 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:49.183347    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:49.397210 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:49.397173    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:49.692793 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:49.692712    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:49.834070 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:49.834032    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:49.929328 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:49.929288    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:50.163394 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.163357    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"]
Jul 27 17:48:50.163988 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.163962    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf320069-b358-4233-a7ef-1a11df5697bf" containerName="maas-api"
Jul 27 17:48:50.163988 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.163987    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf320069-b358-4233-a7ef-1a11df5697bf" containerName="maas-api"
Jul 27 17:48:50.164183 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.164026    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3746261-9cf6-4dd8-a78b-b11afc6e44b9" containerName="payload-processing"
Jul 27 17:48:50.164183 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.164036    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3746261-9cf6-4dd8-a78b-b11afc6e44b9" containerName="payload-processing"
Jul 27 17:48:50.164183 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.164134    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3746261-9cf6-4dd8-a78b-b11afc6e44b9" containerName="payload-processing"
Jul 27 17:48:50.164183 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.164148    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf320069-b358-4233-a7ef-1a11df5697bf" containerName="maas-api"
Jul 27 17:48:50.168923 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.168901    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.172735 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.172709    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d1a612f6-gw-openshift-default-dockercfg-thxsn\""
Jul 27 17:48:50.181725 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.181678    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"]
Jul 27 17:48:50.204565 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.204528    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:50.247873 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.247828    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-certs\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.248039 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.247920    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9a8234-5331-4a9e-91ae-4e679df8a135-istiod-ca-cert\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.248039 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.247992    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-socket\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.248039 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.248011    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-data\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.248039 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.248035    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-podinfo\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.248179 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.248061    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-256g6\" (UniqueName: \"kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-kube-api-access-256g6\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.248179 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.248083    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-credential-socket\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.248179 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.248099    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-envoy\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.248179 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.248122    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-token\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349313 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349282    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-socket\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349313 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349316    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-data\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349544 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349340    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-podinfo\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349544 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349374    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-256g6\" (UniqueName: \"kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-kube-api-access-256g6\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349544 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349407    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-credential-socket\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349544 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349434    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-envoy\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349544 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349467    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-token\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349839 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349567    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-certs\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349839 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349635    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9a8234-5331-4a9e-91ae-4e679df8a135-istiod-ca-cert\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349839 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349790    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-data\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349994 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349911    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-credential-socket\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.349994 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.349983    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-socket\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.350100 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.350021    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-certs\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.350588 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.350567    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9a8234-5331-4a9e-91ae-4e679df8a135-istiod-ca-cert\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.352063 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.352045    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-podinfo\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.352165 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.352142    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-envoy\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.358907 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.358875    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-token\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.359052 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.358989    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-256g6\" (UniqueName: \"kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-kube-api-access-256g6\") pod \"e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") " pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.479766 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.479643    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:50.639056 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.639019    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:50.651141 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.651113    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"]
Jul 27 17:48:50.654155 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:48:50.654118    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9a8234_5331_4a9e_91ae_4e679df8a135.slice/crio-d7f1eb7e2bf192c94cc7531602eca304e3d777f858d23f9a38febc83e0082e4f WatchSource:0}: Error finding container d7f1eb7e2bf192c94cc7531602eca304e3d777f858d23f9a38febc83e0082e4f: Status 404 returned error can't find the container with id d7f1eb7e2bf192c94cc7531602eca304e3d777f858d23f9a38febc83e0082e4f
Jul 27 17:48:50.656613 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.656576    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:48:50.656744 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.656650    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:48:50.656797 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:50.656731    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:48:51.070394 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:51.070354    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" event={"ID":"fc9a8234-5331-4a9e-91ae-4e679df8a135","Type":"ContainerStarted","Data":"871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045"}
Jul 27 17:48:51.070604 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:51.070418    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" event={"ID":"fc9a8234-5331-4a9e-91ae-4e679df8a135","Type":"ContainerStarted","Data":"d7f1eb7e2bf192c94cc7531602eca304e3d777f858d23f9a38febc83e0082e4f"}
Jul 27 17:48:51.095425 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:51.095367    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" podStartSLOduration=1.095350419 podStartE2EDuration="1.095350419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:48:51.093772819 +0000 UTC m=+1668.801921138" watchObservedRunningTime="2026-07-27 17:48:51.095350419 +0000 UTC m=+1668.803498740"
Jul 27 17:48:51.480345 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:51.480258    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:51.485730 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:51.485703    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:52.075419 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.075385    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:52.076841 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.076819    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:48:52.564779 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.564743    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:52.792522 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.792485    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"]
Jul 27 17:48:52.796369 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.796348    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:52.799541 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.799517    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d1a612f6-serving-cert\""
Jul 27 17:48:52.809034 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.809009    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"]
Jul 27 17:48:52.830060 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.829963    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:52.875393 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.875357    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-maas-api-tls\") pod \"maas-api-e2e-migrate-d1a612f6-86797c979-mjss9\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:52.875598 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.875405    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqqj9\" (UniqueName: \"kubernetes.io/projected/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-kube-api-access-jqqj9\") pod \"maas-api-e2e-migrate-d1a612f6-86797c979-mjss9\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:52.875598 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.875503    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-openshift-service-ca\") pod \"maas-api-e2e-migrate-d1a612f6-86797c979-mjss9\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:52.976881 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.976844    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-maas-api-tls\") pod \"maas-api-e2e-migrate-d1a612f6-86797c979-mjss9\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:52.977074 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.976892    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqqj9\" (UniqueName: \"kubernetes.io/projected/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-kube-api-access-jqqj9\") pod \"maas-api-e2e-migrate-d1a612f6-86797c979-mjss9\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:52.977074 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.976953    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-openshift-service-ca\") pod \"maas-api-e2e-migrate-d1a612f6-86797c979-mjss9\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:52.977740 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.977635    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-openshift-service-ca\") pod \"maas-api-e2e-migrate-d1a612f6-86797c979-mjss9\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:52.979708 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.979664    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-maas-api-tls\") pod \"maas-api-e2e-migrate-d1a612f6-86797c979-mjss9\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:52.986369 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:52.986342    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqqj9\" (UniqueName: \"kubernetes.io/projected/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-kube-api-access-jqqj9\") pod \"maas-api-e2e-migrate-d1a612f6-86797c979-mjss9\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:53.019475 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:53.019443    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:53.108443 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:53.108255    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:53.258528 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:53.258502    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"]
Jul 27 17:48:53.260875 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:48:53.260848    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3694cee2_9e04_4ed6_9c84_11576b4b5b0d.slice/crio-a7013157902004439490dfc4d6ae9a9e0c0176624463fcdcd0b32ced80a1c743 WatchSource:0}: Error finding container a7013157902004439490dfc4d6ae9a9e0c0176624463fcdcd0b32ced80a1c743: Status 404 returned error can't find the container with id a7013157902004439490dfc4d6ae9a9e0c0176624463fcdcd0b32ced80a1c743
Jul 27 17:48:53.315107 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:53.315069    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:53.420392 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:53.420295    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:54.087195 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:54.087151    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9" event={"ID":"3694cee2-9e04-4ed6-9c84-11576b4b5b0d","Type":"ContainerStarted","Data":"b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141"}
Jul 27 17:48:54.087596 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:54.087217    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9" event={"ID":"3694cee2-9e04-4ed6-9c84-11576b4b5b0d","Type":"ContainerStarted","Data":"a7013157902004439490dfc4d6ae9a9e0c0176624463fcdcd0b32ced80a1c743"}
Jul 27 17:48:54.087596 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:54.087293    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:48:54.109140 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:48:54.109086    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9" podStartSLOduration=1.682243889 podStartE2EDuration="2.109065357s" totalImagesPullingTime="426.821468ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:48:54.107668079 +0000 UTC m=+1671.815816394" watchObservedRunningTime="2026-07-27 17:48:54.109065357 +0000 UTC m=+1671.817213674"
Jul 27 17:49:00.098582 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:00.098553    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:49:04.142977 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:04.142940    2584 generic.go:356] "Generic (PLEG): container finished" podID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerID="871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045" exitCode=0
Jul 27 17:49:04.143556 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:04.143007    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" event={"ID":"fc9a8234-5331-4a9e-91ae-4e679df8a135","Type":"ContainerDied","Data":"871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045"}
Jul 27 17:49:04.143556 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:04.143424    2584 scope.go:117] "RemoveContainer" containerID="871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045"
Jul 27 17:49:04.144476 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:04.144446    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:49:04.144571 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:04.144534    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:49:04.144635 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:04.144581    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:49:05.150363 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:05.150328    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" event={"ID":"fc9a8234-5331-4a9e-91ae-4e679df8a135","Type":"ContainerStarted","Data":"94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c"}
Jul 27 17:49:05.480959 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:05.480828    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:49:05.482187 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:05.482143    2584 patch_prober.go:28] interesting pod/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 27 17:49:05.482447 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:05.482391    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 27 17:49:06.480778 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:06.480735    2584 patch_prober.go:28] interesting pod/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 27 17:49:06.481434 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:06.481399    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 27 17:49:07.079929 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:07.078794    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:49:07.480390 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:07.480278    2584 patch_prober.go:28] interesting pod/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 27 17:49:07.480390 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:07.480355    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 27 17:49:08.480934 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:08.480885    2584 patch_prober.go:28] interesting pod/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 27 17:49:08.481471 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:08.480955    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 27 17:49:09.484438 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:09.484406    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:49:09.485376 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:09.485352    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:49:27.980618 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:27.980533    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"]
Jul 27 17:49:27.981175 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:27.980802    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9" podUID="3694cee2-9e04-4ed6-9c84-11576b4b5b0d" containerName="maas-api" containerID="cri-o://b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141" gracePeriod=30
Jul 27 17:49:27.991410 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:27.991379    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:28.092569 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.092531    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:28.194651 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.194576    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:28.241638 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.241570    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:49:28.269558 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.269526    2584 generic.go:356] "Generic (PLEG): container finished" podID="3694cee2-9e04-4ed6-9c84-11576b4b5b0d" containerID="b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141" exitCode=0
Jul 27 17:49:28.269783 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.269566    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"
Jul 27 17:49:28.269783 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.269611    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9" event={"ID":"3694cee2-9e04-4ed6-9c84-11576b4b5b0d","Type":"ContainerDied","Data":"b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141"}
Jul 27 17:49:28.269783 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.269672    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9" event={"ID":"3694cee2-9e04-4ed6-9c84-11576b4b5b0d","Type":"ContainerDied","Data":"a7013157902004439490dfc4d6ae9a9e0c0176624463fcdcd0b32ced80a1c743"}
Jul 27 17:49:28.269783 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.269722    2584 scope.go:117] "RemoveContainer" containerID="b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141"
Jul 27 17:49:28.280487 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.280463    2584 scope.go:117] "RemoveContainer" containerID="b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141"
Jul 27 17:49:28.280880 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:49:28.280856    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141\": container with ID starting with b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141 not found: ID does not exist" containerID="b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141"
Jul 27 17:49:28.280987 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.280892    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141"} err="failed to get container status \"b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141\": rpc error: code = NotFound desc = could not find container \"b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141\": container with ID starting with b6dec2ee7578a59cd436352b3095f4ae06eec5584118e66843274a5a70c59141 not found: ID does not exist"
Jul 27 17:49:28.340608 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.340578    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-openshift-service-ca\") pod \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") "
Jul 27 17:49:28.340813 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.340751    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-maas-api-tls\") pod \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") "
Jul 27 17:49:28.340879 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.340819    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqqj9\" (UniqueName: \"kubernetes.io/projected/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-kube-api-access-jqqj9\") pod \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\" (UID: \"3694cee2-9e04-4ed6-9c84-11576b4b5b0d\") "
Jul 27 17:49:28.341036 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.341006    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3694cee2-9e04-4ed6-9c84-11576b4b5b0d" (UID: "3694cee2-9e04-4ed6-9c84-11576b4b5b0d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:49:28.341197 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.341170    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:28.343102 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.343064    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3694cee2-9e04-4ed6-9c84-11576b4b5b0d" (UID: "3694cee2-9e04-4ed6-9c84-11576b4b5b0d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:49:28.343102 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.343091    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-kube-api-access-jqqj9" (OuterVolumeSpecName: "kube-api-access-jqqj9") pod "3694cee2-9e04-4ed6-9c84-11576b4b5b0d" (UID: "3694cee2-9e04-4ed6-9c84-11576b4b5b0d"). InnerVolumeSpecName "kube-api-access-jqqj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:49:28.442536 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.442501    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:28.442536 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.442532    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqqj9\" (UniqueName: \"kubernetes.io/projected/3694cee2-9e04-4ed6-9c84-11576b4b5b0d-kube-api-access-jqqj9\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:28.600648 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.600607    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"]
Jul 27 17:49:28.603005 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.602968    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1a612f6-86797c979-mjss9"]
Jul 27 17:49:28.824171 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:28.824137    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3694cee2-9e04-4ed6-9c84-11576b4b5b0d" path="/var/lib/kubelet/pods/3694cee2-9e04-4ed6-9c84-11576b4b5b0d/volumes"
Jul 27 17:49:32.614651 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:32.614609    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:32.742042 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:32.742003    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"]
Jul 27 17:49:32.742358 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:32.742328    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy" containerID="cri-o://94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c" gracePeriod=30
Jul 27 17:49:32.759769 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:32.759729    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:34.175703 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:34.175651    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:34.296464 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:34.296421    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:36.101943 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.101910    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"]
Jul 27 17:49:36.102354 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.102339    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3694cee2-9e04-4ed6-9c84-11576b4b5b0d" containerName="maas-api"
Jul 27 17:49:36.102399 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.102356    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3694cee2-9e04-4ed6-9c84-11576b4b5b0d" containerName="maas-api"
Jul 27 17:49:36.102438 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.102428    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3694cee2-9e04-4ed6-9c84-11576b4b5b0d" containerName="maas-api"
Jul 27 17:49:36.105734 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.105717    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.108672 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.108650    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-837a3b5f-serving-cert\""
Jul 27 17:49:36.115817 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.115787    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"]
Jul 27 17:49:36.224777 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.224742    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-maas-api-tls\") pod \"maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.224970 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.224809    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cddn\" (UniqueName: \"kubernetes.io/projected/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-kube-api-access-2cddn\") pod \"maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.224970 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.224935    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-openshift-service-ca\") pod \"maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.263112 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.263076    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:36.325733 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.325678    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-openshift-service-ca\") pod \"maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.325937 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.325785    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-maas-api-tls\") pod \"maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.325937 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.325839    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cddn\" (UniqueName: \"kubernetes.io/projected/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-kube-api-access-2cddn\") pod \"maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.325937 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:49:36.325879    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-serving-cert: secret "maas-api-e2e-ait-837a3b5f-serving-cert" not found
Jul 27 17:49:36.326068 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:49:36.325959    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-maas-api-tls podName:4b852813-9cb7-40bf-a0be-0dc9dc6683dd nodeName:}" failed. No retries permitted until 2026-07-27 17:49:36.825936619 +0000 UTC m=+1714.534084915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-maas-api-tls") pod "maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp" (UID: "4b852813-9cb7-40bf-a0be-0dc9dc6683dd") : secret "maas-api-e2e-ait-837a3b5f-serving-cert" not found
Jul 27 17:49:36.326265 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.326246    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-openshift-service-ca\") pod \"maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.337700 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.337651    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cddn\" (UniqueName: \"kubernetes.io/projected/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-kube-api-access-2cddn\") pod \"maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.364254 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.364164    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:36.674411 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.674310    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:36.828723 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.828678    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:36.831480 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.831444    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-maas-api-tls\") pod \"maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.834402 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.834379    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-maas-api-tls\") pod \"maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:36.920196 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:36.920161    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:37.019273 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:37.019173    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:37.166769 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:37.166742    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"]
Jul 27 17:49:37.169472 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:49:37.169445    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b852813_9cb7_40bf_a0be_0dc9dc6683dd.slice/crio-4ab56bd900ebf5d806a96d768ae229834b870702be57456403553f27f06c3ca5 WatchSource:0}: Error finding container 4ab56bd900ebf5d806a96d768ae229834b870702be57456403553f27f06c3ca5: Status 404 returned error can't find the container with id 4ab56bd900ebf5d806a96d768ae229834b870702be57456403553f27f06c3ca5
Jul 27 17:49:37.323061 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:37.323018    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp" event={"ID":"4b852813-9cb7-40bf-a0be-0dc9dc6683dd","Type":"ContainerStarted","Data":"4ab56bd900ebf5d806a96d768ae229834b870702be57456403553f27f06c3ca5"}
Jul 27 17:49:38.010750 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.010721    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:49:38.146573 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.146539    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-data\") pod \"fc9a8234-5331-4a9e-91ae-4e679df8a135\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") "
Jul 27 17:49:38.146573 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.146578    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-socket\") pod \"fc9a8234-5331-4a9e-91ae-4e679df8a135\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") "
Jul 27 17:49:38.146893 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.146603    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-256g6\" (UniqueName: \"kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-kube-api-access-256g6\") pod \"fc9a8234-5331-4a9e-91ae-4e679df8a135\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") "
Jul 27 17:49:38.146893 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.146623    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-token\") pod \"fc9a8234-5331-4a9e-91ae-4e679df8a135\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") "
Jul 27 17:49:38.146893 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.146660    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-podinfo\") pod \"fc9a8234-5331-4a9e-91ae-4e679df8a135\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") "
Jul 27 17:49:38.146893 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.146746    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-certs\") pod \"fc9a8234-5331-4a9e-91ae-4e679df8a135\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") "
Jul 27 17:49:38.146893 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.146818    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9a8234-5331-4a9e-91ae-4e679df8a135-istiod-ca-cert\") pod \"fc9a8234-5331-4a9e-91ae-4e679df8a135\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") "
Jul 27 17:49:38.146893 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.146848    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-credential-socket\") pod \"fc9a8234-5331-4a9e-91ae-4e679df8a135\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") "
Jul 27 17:49:38.147281 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.146908    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-envoy\") pod \"fc9a8234-5331-4a9e-91ae-4e679df8a135\" (UID: \"fc9a8234-5331-4a9e-91ae-4e679df8a135\") "
Jul 27 17:49:38.147281 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.146984    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc9a8234-5331-4a9e-91ae-4e679df8a135" (UID: "fc9a8234-5331-4a9e-91ae-4e679df8a135"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:49:38.147281 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.147226    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc9a8234-5331-4a9e-91ae-4e679df8a135" (UID: "fc9a8234-5331-4a9e-91ae-4e679df8a135"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:49:38.147435 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.147279    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-socket\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:38.147490 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.147424    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc9a8234-5331-4a9e-91ae-4e679df8a135-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc9a8234-5331-4a9e-91ae-4e679df8a135" (UID: "fc9a8234-5331-4a9e-91ae-4e679df8a135"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:49:38.147644 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.147622    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc9a8234-5331-4a9e-91ae-4e679df8a135" (UID: "fc9a8234-5331-4a9e-91ae-4e679df8a135"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:49:38.147741 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.147658    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc9a8234-5331-4a9e-91ae-4e679df8a135" (UID: "fc9a8234-5331-4a9e-91ae-4e679df8a135"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:49:38.149245 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.149220    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc9a8234-5331-4a9e-91ae-4e679df8a135" (UID: "fc9a8234-5331-4a9e-91ae-4e679df8a135"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:49:38.149353 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.149256    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-kube-api-access-256g6" (OuterVolumeSpecName: "kube-api-access-256g6") pod "fc9a8234-5331-4a9e-91ae-4e679df8a135" (UID: "fc9a8234-5331-4a9e-91ae-4e679df8a135"). InnerVolumeSpecName "kube-api-access-256g6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:49:38.149353 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.149254    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc9a8234-5331-4a9e-91ae-4e679df8a135" (UID: "fc9a8234-5331-4a9e-91ae-4e679df8a135"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 17:49:38.149561 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.149545    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc9a8234-5331-4a9e-91ae-4e679df8a135" (UID: "fc9a8234-5331-4a9e-91ae-4e679df8a135"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:49:38.248205 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.248113    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-workload-certs\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:38.248205 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.248144    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9a8234-5331-4a9e-91ae-4e679df8a135-istiod-ca-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:38.248205 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.248153    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-credential-socket\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:38.248205 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.248163    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-envoy\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:38.248205 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.248171    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-data\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:38.248205 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.248179    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-256g6\" (UniqueName: \"kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-kube-api-access-256g6\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:38.248205 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.248187    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-token\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:38.248205 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.248195    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9a8234-5331-4a9e-91ae-4e679df8a135-istio-podinfo\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:49:38.330036 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.330000    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp" event={"ID":"4b852813-9cb7-40bf-a0be-0dc9dc6683dd","Type":"ContainerStarted","Data":"3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524"}
Jul 27 17:49:38.330264 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.330105    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:49:38.331829 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.331808    2584 generic.go:356] "Generic (PLEG): container finished" podID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerID="94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c" exitCode=0
Jul 27 17:49:38.331960 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.331857    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"
Jul 27 17:49:38.331960 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.331900    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" event={"ID":"fc9a8234-5331-4a9e-91ae-4e679df8a135","Type":"ContainerDied","Data":"94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c"}
Jul 27 17:49:38.332075 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.331954    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl" event={"ID":"fc9a8234-5331-4a9e-91ae-4e679df8a135","Type":"ContainerDied","Data":"d7f1eb7e2bf192c94cc7531602eca304e3d777f858d23f9a38febc83e0082e4f"}
Jul 27 17:49:38.332075 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.331990    2584 scope.go:117] "RemoveContainer" containerID="94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c"
Jul 27 17:49:38.343944 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.343913    2584 scope.go:117] "RemoveContainer" containerID="871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045"
Jul 27 17:49:38.353259 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.353241    2584 scope.go:117] "RemoveContainer" containerID="94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c"
Jul 27 17:49:38.353539 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:49:38.353519    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c\": container with ID starting with 94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c not found: ID does not exist" containerID="94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c"
Jul 27 17:49:38.353614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.353552    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c"} err="failed to get container status \"94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c\": rpc error: code = NotFound desc = could not find container \"94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c\": container with ID starting with 94997b856acf2f8c6f7da0662ba0f98f059398396c67d008b0af0dfc0b6caa5c not found: ID does not exist"
Jul 27 17:49:38.353614 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.353579    2584 scope.go:117] "RemoveContainer" containerID="871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045"
Jul 27 17:49:38.353909 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:49:38.353891    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045\": container with ID starting with 871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045 not found: ID does not exist" containerID="871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045"
Jul 27 17:49:38.353953 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.353917    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045"} err="failed to get container status \"871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045\": rpc error: code = NotFound desc = could not find container \"871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045\": container with ID starting with 871cabb417cb68297dff378f793709763a0f750cda20d6272441ce1fa043a045 not found: ID does not exist"
Jul 27 17:49:38.356468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.356425    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp" podStartSLOduration=1.7943434200000001 podStartE2EDuration="2.356409105s" totalImagesPullingTime="562.065685ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:49:38.354777248 +0000 UTC m=+1716.062925567" watchObservedRunningTime="2026-07-27 17:49:38.356409105 +0000 UTC m=+1716.064557425"
Jul 27 17:49:38.372006 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.371978    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"]
Jul 27 17:49:38.376633 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.376601    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d1a612f6-gw-openshift-default-54b99975d8-4m4gl"]
Jul 27 17:49:38.823745 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:38.823712    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" path="/var/lib/kubelet/pods/fc9a8234-5331-4a9e-91ae-4e679df8a135/volumes"
Jul 27 17:49:44.343413 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:49:44.343380    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:50:12.214236 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.214195    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"]
Jul 27 17:50:12.214792 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.214464    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp" podUID="4b852813-9cb7-40bf-a0be-0dc9dc6683dd" containerName="maas-api" containerID="cri-o://3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524" gracePeriod=30
Jul 27 17:50:12.238829 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.238800    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:12.312197 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.312161    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:12.375902 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.375855    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:12.472261 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.472202    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:50:12.511405 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.511376    2584 generic.go:356] "Generic (PLEG): container finished" podID="4b852813-9cb7-40bf-a0be-0dc9dc6683dd" containerID="3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524" exitCode=0
Jul 27 17:50:12.511576 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.511429    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp" event={"ID":"4b852813-9cb7-40bf-a0be-0dc9dc6683dd","Type":"ContainerDied","Data":"3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524"}
Jul 27 17:50:12.511576 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.511445    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"
Jul 27 17:50:12.511576 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.511473    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp" event={"ID":"4b852813-9cb7-40bf-a0be-0dc9dc6683dd","Type":"ContainerDied","Data":"4ab56bd900ebf5d806a96d768ae229834b870702be57456403553f27f06c3ca5"}
Jul 27 17:50:12.511576 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.511496    2584 scope.go:117] "RemoveContainer" containerID="3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524"
Jul 27 17:50:12.521918 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.521900    2584 scope.go:117] "RemoveContainer" containerID="3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524"
Jul 27 17:50:12.522212 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:50:12.522190    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524\": container with ID starting with 3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524 not found: ID does not exist" containerID="3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524"
Jul 27 17:50:12.522315 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.522220    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524"} err="failed to get container status \"3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524\": rpc error: code = NotFound desc = could not find container \"3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524\": container with ID starting with 3d4cd59666667a3b28b6974a9d0c740f614fc697ae6a9002af07869c7081f524 not found: ID does not exist"
Jul 27 17:50:12.586808 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.586773    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cddn\" (UniqueName: \"kubernetes.io/projected/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-kube-api-access-2cddn\") pod \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") "
Jul 27 17:50:12.586979 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.586857    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-openshift-service-ca\") pod \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") "
Jul 27 17:50:12.587051 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.587027    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-maas-api-tls\") pod \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\" (UID: \"4b852813-9cb7-40bf-a0be-0dc9dc6683dd\") "
Jul 27 17:50:12.587198 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.587162    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b852813-9cb7-40bf-a0be-0dc9dc6683dd" (UID: "4b852813-9cb7-40bf-a0be-0dc9dc6683dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:50:12.587419 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.587404    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:50:12.589068 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.589045    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-kube-api-access-2cddn" (OuterVolumeSpecName: "kube-api-access-2cddn") pod "4b852813-9cb7-40bf-a0be-0dc9dc6683dd" (UID: "4b852813-9cb7-40bf-a0be-0dc9dc6683dd"). InnerVolumeSpecName "kube-api-access-2cddn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:50:12.589166 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.589148    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b852813-9cb7-40bf-a0be-0dc9dc6683dd" (UID: "4b852813-9cb7-40bf-a0be-0dc9dc6683dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:50:12.688298 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.688267    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cddn\" (UniqueName: \"kubernetes.io/projected/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-kube-api-access-2cddn\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:50:12.688298 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.688295    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b852813-9cb7-40bf-a0be-0dc9dc6683dd-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:50:12.834911 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.834880    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"]
Jul 27 17:50:12.836659 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:12.836638    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-837a3b5f-d56d8d98-85wvp"]
Jul 27 17:50:14.824512 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:14.824474    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b852813-9cb7-40bf-a0be-0dc9dc6683dd" path="/var/lib/kubelet/pods/4b852813-9cb7-40bf-a0be-0dc9dc6683dd/volumes"
Jul 27 17:50:16.726984 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:16.726950    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:17.866283 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:17.866246    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:18.089875 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.089834    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:18.323952 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.323910    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:18.569865 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.569836    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"]
Jul 27 17:50:18.570405 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.570360    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b852813-9cb7-40bf-a0be-0dc9dc6683dd" containerName="maas-api"
Jul 27 17:50:18.570405 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.570386    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b852813-9cb7-40bf-a0be-0dc9dc6683dd" containerName="maas-api"
Jul 27 17:50:18.570633 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.570433    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy"
Jul 27 17:50:18.570633 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.570444    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy"
Jul 27 17:50:18.570633 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.570455    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy"
Jul 27 17:50:18.570633 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.570463    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy"
Jul 27 17:50:18.570633 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.570551    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy"
Jul 27 17:50:18.570633 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.570564    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc9a8234-5331-4a9e-91ae-4e679df8a135" containerName="istio-proxy"
Jul 27 17:50:18.570633 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.570579    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b852813-9cb7-40bf-a0be-0dc9dc6683dd" containerName="maas-api"
Jul 27 17:50:18.575197 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.575138    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.578128 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.578106    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-d5de8abc-openshift-default-dockercfg-chlsj\""
Jul 27 17:50:18.586932 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.586903    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"]
Jul 27 17:50:18.590211 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.590179    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:18.660438 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.660400    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg5qs\" (UniqueName: \"kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-kube-api-access-rg5qs\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.660792 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.660742    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-envoy\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.660952 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.660935    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-token\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.661106 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.661090    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-socket\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.661258 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.661240    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-credential-socket\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.661392 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.661357    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-data\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.661525 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.661394    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-certs\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.661525 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.661445    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-podinfo\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.661525 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.661515    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istiod-ca-cert\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762163 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762108    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg5qs\" (UniqueName: \"kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-kube-api-access-rg5qs\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762382 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762179    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-envoy\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762382 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762212    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-token\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762382 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762247    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-socket\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762382 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762293    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-credential-socket\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762382 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762361    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-data\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762716 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762412    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-certs\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762716 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762453    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-podinfo\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762716 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762522    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istiod-ca-cert\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762716 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762680    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-socket\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762959 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762789    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-credential-socket\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762959 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762827    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-certs\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.762959 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.762918    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-data\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.763221 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.763203    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istiod-ca-cert\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.764732 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.764714    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-envoy\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.765092 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.765073    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-podinfo\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.771038 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.771011    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-token\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.771186 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.771169    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg5qs\" (UniqueName: \"kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-kube-api-access-rg5qs\") pod \"e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") " pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.886196 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.886097    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:18.956789 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:18.956752    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:19.039415 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:19.039298    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"]
Jul 27 17:50:19.042432 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:50:19.042397    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbed79d33_ce8a_4f12_b0d8_26dfc3f0a658.slice/crio-6d141e33d359240ac11750cb2a424e1be1209e4e92de5842f9f614b4b1ddbbb3 WatchSource:0}: Error finding container 6d141e33d359240ac11750cb2a424e1be1209e4e92de5842f9f614b4b1ddbbb3: Status 404 returned error can't find the container with id 6d141e33d359240ac11750cb2a424e1be1209e4e92de5842f9f614b4b1ddbbb3
Jul 27 17:50:19.044768 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:19.044725    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:50:19.044847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:19.044829    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:50:19.044896 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:19.044879    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:50:19.550492 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:19.550460    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" event={"ID":"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658","Type":"ContainerStarted","Data":"c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746"}
Jul 27 17:50:19.550703 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:19.550515    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" event={"ID":"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658","Type":"ContainerStarted","Data":"6d141e33d359240ac11750cb2a424e1be1209e4e92de5842f9f614b4b1ddbbb3"}
Jul 27 17:50:19.572317 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:19.572266    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" podStartSLOduration=1.572250309 podStartE2EDuration="1.572250309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:50:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:50:19.570970109 +0000 UTC m=+1757.279118466" watchObservedRunningTime="2026-07-27 17:50:19.572250309 +0000 UTC m=+1757.280398628"
Jul 27 17:50:19.886320 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:19.886283    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:20.892506 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:20.892443    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:20.923104 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:20.923037    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:21.000261 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.000224    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"]
Jul 27 17:50:21.002726 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.002667    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:21.004894 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.004871    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.007927 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.007898    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-d5de8abc-serving-cert\""
Jul 27 17:50:21.013495 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.013458    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"]
Jul 27 17:50:21.073873 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.073841    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:21.085042 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.085007    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5spwj\" (UniqueName: \"kubernetes.io/projected/abadb848-bc03-4bd2-8f4e-5b1725a49def-kube-api-access-5spwj\") pod \"maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.085219 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.085093    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abadb848-bc03-4bd2-8f4e-5b1725a49def-maas-api-tls\") pod \"maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.085219 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.085195    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abadb848-bc03-4bd2-8f4e-5b1725a49def-openshift-service-ca\") pod \"maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.185911 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.185808    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abadb848-bc03-4bd2-8f4e-5b1725a49def-openshift-service-ca\") pod \"maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.185911 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.185893    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5spwj\" (UniqueName: \"kubernetes.io/projected/abadb848-bc03-4bd2-8f4e-5b1725a49def-kube-api-access-5spwj\") pod \"maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.186149 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.185952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abadb848-bc03-4bd2-8f4e-5b1725a49def-maas-api-tls\") pod \"maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.186149 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:50:21.186085    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-serving-cert: secret "maas-api-e2e-derive-d5de8abc-serving-cert" not found
Jul 27 17:50:21.186272 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:50:21.186163    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abadb848-bc03-4bd2-8f4e-5b1725a49def-maas-api-tls podName:abadb848-bc03-4bd2-8f4e-5b1725a49def nodeName:}" failed. No retries permitted until 2026-07-27 17:50:21.686141907 +0000 UTC m=+1759.394290230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/abadb848-bc03-4bd2-8f4e-5b1725a49def-maas-api-tls") pod "maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j" (UID: "abadb848-bc03-4bd2-8f4e-5b1725a49def") : secret "maas-api-e2e-derive-d5de8abc-serving-cert" not found
Jul 27 17:50:21.186666 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.186642    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abadb848-bc03-4bd2-8f4e-5b1725a49def-openshift-service-ca\") pod \"maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.198807 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.198773    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5spwj\" (UniqueName: \"kubernetes.io/projected/abadb848-bc03-4bd2-8f4e-5b1725a49def-kube-api-access-5spwj\") pod \"maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.404416 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.404375    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:21.561277 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.561247    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:21.562229 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.562208    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:21.629503 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.629467    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:21.691362 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.691325    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abadb848-bc03-4bd2-8f4e-5b1725a49def-maas-api-tls\") pod \"maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.694175 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.694147    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abadb848-bc03-4bd2-8f4e-5b1725a49def-maas-api-tls\") pod \"maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:21.729317 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.729272    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:21.918437 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:21.918333    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:22.066678 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:22.066653    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"]
Jul 27 17:50:22.068984 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:50:22.068939    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabadb848_bc03_4bd2_8f4e_5b1725a49def.slice/crio-a5e2fa10a40e61b5c044edca5b252ef1232369e79941e2b1e0bc082288232f59 WatchSource:0}: Error finding container a5e2fa10a40e61b5c044edca5b252ef1232369e79941e2b1e0bc082288232f59: Status 404 returned error can't find the container with id a5e2fa10a40e61b5c044edca5b252ef1232369e79941e2b1e0bc082288232f59
Jul 27 17:50:22.567563 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:22.567527    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j" event={"ID":"abadb848-bc03-4bd2-8f4e-5b1725a49def","Type":"ContainerStarted","Data":"a5e2fa10a40e61b5c044edca5b252ef1232369e79941e2b1e0bc082288232f59"}
Jul 27 17:50:23.574268 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:23.574233    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j" event={"ID":"abadb848-bc03-4bd2-8f4e-5b1725a49def","Type":"ContainerStarted","Data":"b6a540f5be05ea84495e0d974403b4d82df7d26f87e666262d3c0cf40c6b9271"}
Jul 27 17:50:23.574746 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:23.574441    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:23.593341 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:23.593287    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j" podStartSLOduration=2.931502536 podStartE2EDuration="3.593269352s" totalImagesPullingTime="661.766816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:50:23.592644952 +0000 UTC m=+1761.300793271" watchObservedRunningTime="2026-07-27 17:50:23.593269352 +0000 UTC m=+1761.301417670"
Jul 27 17:50:29.591085 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:29.591043    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:35.637379 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:35.637348    2584 generic.go:356] "Generic (PLEG): container finished" podID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerID="c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746" exitCode=0
Jul 27 17:50:35.637825 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:35.637429    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" event={"ID":"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658","Type":"ContainerDied","Data":"c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746"}
Jul 27 17:50:35.637887 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:35.637872    2584 scope.go:117] "RemoveContainer" containerID="c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746"
Jul 27 17:50:35.638954 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:35.638926    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:50:35.639032 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:35.639017    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:50:35.639074 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:35.639061    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:50:36.561570 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:36.561539    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:36.644660 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:36.644624    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" event={"ID":"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658","Type":"ContainerStarted","Data":"7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2"}
Jul 27 17:50:36.887355 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:36.887248    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:36.888571 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:36.888543    2584 patch_prober.go:28] interesting pod/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 27 17:50:36.888822 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:36.888796    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 27 17:50:37.886540 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:37.886499    2584 patch_prober.go:28] interesting pod/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 27 17:50:37.887160 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:37.887125    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 27 17:50:38.887623 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:38.887569    2584 patch_prober.go:28] interesting pod/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 27 17:50:38.888171 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:38.887651    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 27 17:50:39.887122 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:39.887082    2584 patch_prober.go:28] interesting pod/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 27 17:50:39.887431 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:39.887400    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 27 17:50:40.891660 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:40.891626    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:40.892108 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:40.892068    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:40.892762 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:40.892740    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:50:56.549096 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.549012    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"]
Jul 27 17:50:56.549626 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.549242    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j" podUID="abadb848-bc03-4bd2-8f4e-5b1725a49def" containerName="maas-api" containerID="cri-o://b6a540f5be05ea84495e0d974403b4d82df7d26f87e666262d3c0cf40c6b9271" gracePeriod=30
Jul 27 17:50:56.558883 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.558852    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:56.635329 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.635293    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:56.746854 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.746812    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:56.752239 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.752210    2584 generic.go:356] "Generic (PLEG): container finished" podID="abadb848-bc03-4bd2-8f4e-5b1725a49def" containerID="b6a540f5be05ea84495e0d974403b4d82df7d26f87e666262d3c0cf40c6b9271" exitCode=0
Jul 27 17:50:56.752402 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.752281    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j" event={"ID":"abadb848-bc03-4bd2-8f4e-5b1725a49def","Type":"ContainerDied","Data":"b6a540f5be05ea84495e0d974403b4d82df7d26f87e666262d3c0cf40c6b9271"}
Jul 27 17:50:56.813206 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.813178    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:56.984354 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.984314    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abadb848-bc03-4bd2-8f4e-5b1725a49def-openshift-service-ca\") pod \"abadb848-bc03-4bd2-8f4e-5b1725a49def\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") "
Jul 27 17:50:56.984354 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.984360    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abadb848-bc03-4bd2-8f4e-5b1725a49def-maas-api-tls\") pod \"abadb848-bc03-4bd2-8f4e-5b1725a49def\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") "
Jul 27 17:50:56.984598 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.984470    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5spwj\" (UniqueName: \"kubernetes.io/projected/abadb848-bc03-4bd2-8f4e-5b1725a49def-kube-api-access-5spwj\") pod \"abadb848-bc03-4bd2-8f4e-5b1725a49def\" (UID: \"abadb848-bc03-4bd2-8f4e-5b1725a49def\") "
Jul 27 17:50:56.984746 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.984669    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abadb848-bc03-4bd2-8f4e-5b1725a49def-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "abadb848-bc03-4bd2-8f4e-5b1725a49def" (UID: "abadb848-bc03-4bd2-8f4e-5b1725a49def"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:50:56.986822 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.986797    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abadb848-bc03-4bd2-8f4e-5b1725a49def-kube-api-access-5spwj" (OuterVolumeSpecName: "kube-api-access-5spwj") pod "abadb848-bc03-4bd2-8f4e-5b1725a49def" (UID: "abadb848-bc03-4bd2-8f4e-5b1725a49def"). InnerVolumeSpecName "kube-api-access-5spwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:50:56.986930 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:56.986843    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abadb848-bc03-4bd2-8f4e-5b1725a49def-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "abadb848-bc03-4bd2-8f4e-5b1725a49def" (UID: "abadb848-bc03-4bd2-8f4e-5b1725a49def"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:50:57.085929 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:57.085841    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abadb848-bc03-4bd2-8f4e-5b1725a49def-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:50:57.085929 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:57.085873    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5spwj\" (UniqueName: \"kubernetes.io/projected/abadb848-bc03-4bd2-8f4e-5b1725a49def-kube-api-access-5spwj\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:50:57.085929 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:57.085883    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abadb848-bc03-4bd2-8f4e-5b1725a49def-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:50:57.758541 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:57.758513    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"
Jul 27 17:50:57.758980 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:57.758513    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j" event={"ID":"abadb848-bc03-4bd2-8f4e-5b1725a49def","Type":"ContainerDied","Data":"a5e2fa10a40e61b5c044edca5b252ef1232369e79941e2b1e0bc082288232f59"}
Jul 27 17:50:57.758980 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:57.758650    2584 scope.go:117] "RemoveContainer" containerID="b6a540f5be05ea84495e0d974403b4d82df7d26f87e666262d3c0cf40c6b9271"
Jul 27 17:50:57.780615 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:57.780586    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"]
Jul 27 17:50:57.785911 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:57.785879    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5de8abc-59bcbd4d8b-2q77j"]
Jul 27 17:50:58.823492 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:50:58.823444    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abadb848-bc03-4bd2-8f4e-5b1725a49def" path="/var/lib/kubelet/pods/abadb848-bc03-4bd2-8f4e-5b1725a49def/volumes"
Jul 27 17:51:00.836825 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:00.836783    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:00.965221 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:00.965186    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"]
Jul 27 17:51:00.965468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:00.965414    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy" containerID="cri-o://7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2" gracePeriod=30
Jul 27 17:51:00.990258 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:00.990224    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:01.216341 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:01.216236    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:01.657041 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:01.656983    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:01.756938 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:01.756881    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:01.866832 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:01.866800    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:03.272483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:03.272382    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:06.228990 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.228923    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:51:06.281038 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281005    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istiod-ca-cert\") pod \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") "
Jul 27 17:51:06.281038 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281045    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg5qs\" (UniqueName: \"kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-kube-api-access-rg5qs\") pod \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") "
Jul 27 17:51:06.281249 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281085    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-token\") pod \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") "
Jul 27 17:51:06.281249 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281105    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-socket\") pod \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") "
Jul 27 17:51:06.281249 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281159    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-envoy\") pod \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") "
Jul 27 17:51:06.281249 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281201    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-certs\") pod \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") "
Jul 27 17:51:06.281249 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281228    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-credential-socket\") pod \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") "
Jul 27 17:51:06.281496 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281257    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-podinfo\") pod \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") "
Jul 27 17:51:06.281496 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281304    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-data\") pod \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\" (UID: \"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658\") "
Jul 27 17:51:06.281496 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281457    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" (UID: "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:51:06.281663 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281493    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" (UID: "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:51:06.281663 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281569    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" (UID: "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:51:06.281800 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281721    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" (UID: "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:51:06.281858 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281801    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-socket\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:51:06.281858 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281827    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-workload-certs\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:51:06.281858 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.281843    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istiod-ca-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:51:06.282612 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.282586    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-data" (OuterVolumeSpecName: "istio-data") pod "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" (UID: "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:51:06.283727 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.283677    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" (UID: "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:51:06.283727 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.283708    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" (UID: "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 17:51:06.284000 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.283983    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-token" (OuterVolumeSpecName: "istio-token") pod "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" (UID: "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:51:06.284049 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.284028    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-kube-api-access-rg5qs" (OuterVolumeSpecName: "kube-api-access-rg5qs") pod "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" (UID: "bed79d33-ce8a-4f12-b0d8-26dfc3f0a658"). InnerVolumeSpecName "kube-api-access-rg5qs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:51:06.383266 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.383235    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-credential-socket\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:51:06.383266 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.383265    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-podinfo\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:51:06.383266 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.383275    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-data\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:51:06.383504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.383306    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg5qs\" (UniqueName: \"kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-kube-api-access-rg5qs\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:51:06.383504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.383317    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-token\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:51:06.383504 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.383325    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658-istio-envoy\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:51:06.814618 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.814585    2584 generic.go:356] "Generic (PLEG): container finished" podID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerID="7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2" exitCode=0
Jul 27 17:51:06.814835 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.814645    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" event={"ID":"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658","Type":"ContainerDied","Data":"7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2"}
Jul 27 17:51:06.814835 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.814655    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"
Jul 27 17:51:06.814835 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.814743    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw" event={"ID":"bed79d33-ce8a-4f12-b0d8-26dfc3f0a658","Type":"ContainerDied","Data":"6d141e33d359240ac11750cb2a424e1be1209e4e92de5842f9f614b4b1ddbbb3"}
Jul 27 17:51:06.814835 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.814767    2584 scope.go:117] "RemoveContainer" containerID="7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2"
Jul 27 17:51:06.825140 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.825116    2584 scope.go:117] "RemoveContainer" containerID="c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746"
Jul 27 17:51:06.833896 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.833873    2584 scope.go:117] "RemoveContainer" containerID="7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2"
Jul 27 17:51:06.834192 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:51:06.834172    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2\": container with ID starting with 7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2 not found: ID does not exist" containerID="7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2"
Jul 27 17:51:06.834265 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.834207    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2"} err="failed to get container status \"7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2\": rpc error: code = NotFound desc = could not find container \"7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2\": container with ID starting with 7f863c7c73daf7be5ad0bfccd08fcdac92989e5a78813d8080d70c8ea10a49e2 not found: ID does not exist"
Jul 27 17:51:06.834265 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.834227    2584 scope.go:117] "RemoveContainer" containerID="c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746"
Jul 27 17:51:06.834493 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:51:06.834472    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746\": container with ID starting with c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746 not found: ID does not exist" containerID="c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746"
Jul 27 17:51:06.834550 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.834502    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746"} err="failed to get container status \"c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746\": rpc error: code = NotFound desc = could not find container \"c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746\": container with ID starting with c3583710cde8343fdb626ea29532cc8dfa93cee143f28bbf4e48a6f8a110e746 not found: ID does not exist"
Jul 27 17:51:06.838600 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.838576    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"]
Jul 27 17:51:06.841837 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:06.841812    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-d5de8abc-openshift-default-6db7f4fb56-nwfkw"]
Jul 27 17:51:08.824258 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:08.824223    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" path="/var/lib/kubelet/pods/bed79d33-ce8a-4f12-b0d8-26dfc3f0a658/volumes"
Jul 27 17:51:09.545259 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:09.545216    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:10.024812 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:10.024770    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:10.124286 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:10.124245    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:10.220059 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:10.220019    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:36.188407 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:36.188372    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:42.874847 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:42.874809    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:42.941760 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:42.941714    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:43.055637 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:43.055598    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:43.144908 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:43.144805    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:48.172513 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:48.172475    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:48.887397 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:48.887356    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:58.950806 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:51:58.950764    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:04.736998 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:04.736955    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:04.847185 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:04.847147    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:04.938555 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:04.938518    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:15.865824 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:15.865787    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:16.441283 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:16.441241    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:22.956213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:22.956123    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:23.059306 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:23.059268    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:23.158921 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:23.158879    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:23.298789 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:23.298676    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:24.060654 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:24.060614    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:24.160611 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:24.160575    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:24.260942 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:24.260907    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:26.391755 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:26.391714    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:32.620660 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:32.620613    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:32.722341 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:32.722292    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:33.545310 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:33.545275    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:39.194885 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:39.194846    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:40.868612 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:40.868578    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:42.615233 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.615196    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:42.713115 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.713082    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:42.869082 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.868984    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"]
Jul 27 17:52:42.869402 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.869388    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abadb848-bc03-4bd2-8f4e-5b1725a49def" containerName="maas-api"
Jul 27 17:52:42.869402 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.869402    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="abadb848-bc03-4bd2-8f4e-5b1725a49def" containerName="maas-api"
Jul 27 17:52:42.869529 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.869415    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy"
Jul 27 17:52:42.869529 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.869421    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy"
Jul 27 17:52:42.869529 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.869427    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy"
Jul 27 17:52:42.869529 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.869433    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy"
Jul 27 17:52:42.869529 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.869488    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy"
Jul 27 17:52:42.869529 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.869498    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="abadb848-bc03-4bd2-8f4e-5b1725a49def" containerName="maas-api"
Jul 27 17:52:42.872603 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.872567    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:42.876185 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.876165    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3852632a-serving-cert\""
Jul 27 17:52:42.884950 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.884922    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"]
Jul 27 17:52:42.918375 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.918342    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:42.933572 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.933534    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpjwr\" (UniqueName: \"kubernetes.io/projected/cc8e992b-b37b-4416-aa2a-f182c0701914-kube-api-access-rpjwr\") pod \"maas-api-e2e-mt-3852632a-859fff7c74-ngd4p\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:42.933784 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.933725    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8e992b-b37b-4416-aa2a-f182c0701914-openshift-service-ca\") pod \"maas-api-e2e-mt-3852632a-859fff7c74-ngd4p\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:42.934090 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:42.934058    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8e992b-b37b-4416-aa2a-f182c0701914-maas-api-tls\") pod \"maas-api-e2e-mt-3852632a-859fff7c74-ngd4p\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:43.034194 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.034162    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:43.034662 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.034643    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8e992b-b37b-4416-aa2a-f182c0701914-openshift-service-ca\") pod \"maas-api-e2e-mt-3852632a-859fff7c74-ngd4p\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:43.034759 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.034743    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8e992b-b37b-4416-aa2a-f182c0701914-maas-api-tls\") pod \"maas-api-e2e-mt-3852632a-859fff7c74-ngd4p\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:43.034814 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.034779    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpjwr\" (UniqueName: \"kubernetes.io/projected/cc8e992b-b37b-4416-aa2a-f182c0701914-kube-api-access-rpjwr\") pod \"maas-api-e2e-mt-3852632a-859fff7c74-ngd4p\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:43.035420 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.035391    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8e992b-b37b-4416-aa2a-f182c0701914-openshift-service-ca\") pod \"maas-api-e2e-mt-3852632a-859fff7c74-ngd4p\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:43.037410 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.037387    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8e992b-b37b-4416-aa2a-f182c0701914-maas-api-tls\") pod \"maas-api-e2e-mt-3852632a-859fff7c74-ngd4p\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:43.043572 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.043541    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpjwr\" (UniqueName: \"kubernetes.io/projected/cc8e992b-b37b-4416-aa2a-f182c0701914-kube-api-access-rpjwr\") pod \"maas-api-e2e-mt-3852632a-859fff7c74-ngd4p\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:43.184412 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.184325    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:43.332819 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.332785    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"]
Jul 27 17:52:43.336090 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:52:43.336059    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8e992b_b37b_4416_aa2a_f182c0701914.slice/crio-9275f7d9153ebf814803368e0b83ea2f4e0234bb1fe444c02e8d89218bb998a3 WatchSource:0}: Error finding container 9275f7d9153ebf814803368e0b83ea2f4e0234bb1fe444c02e8d89218bb998a3: Status 404 returned error can't find the container with id 9275f7d9153ebf814803368e0b83ea2f4e0234bb1fe444c02e8d89218bb998a3
Jul 27 17:52:43.503125 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.503031    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:43.604886 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:43.604843    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:44.312223 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:44.312186    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p" event={"ID":"cc8e992b-b37b-4416-aa2a-f182c0701914","Type":"ContainerStarted","Data":"db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e"}
Jul 27 17:52:44.312624 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:44.312243    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p" event={"ID":"cc8e992b-b37b-4416-aa2a-f182c0701914","Type":"ContainerStarted","Data":"9275f7d9153ebf814803368e0b83ea2f4e0234bb1fe444c02e8d89218bb998a3"}
Jul 27 17:52:44.312624 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:44.312265    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:44.330792 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:44.330739    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p" podStartSLOduration=1.811506482 podStartE2EDuration="2.330721281s" totalImagesPullingTime="519.214799ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:52:44.32849136 +0000 UTC m=+1902.036639678" watchObservedRunningTime="2026-07-27 17:52:44.330721281 +0000 UTC m=+1902.038869599"
Jul 27 17:52:47.059627 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:47.059592    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:47.711939 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:47.711901    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:47.812320 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:47.812287    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:48.135463 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:48.135428    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:50.323365 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:50.323333    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:52:54.020716 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:54.020645    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:54.121311 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:52:54.121247    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:04.841165 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:04.841124    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"]
Jul 27 17:53:04.841589 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:04.841369    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p" podUID="cc8e992b-b37b-4416-aa2a-f182c0701914" containerName="maas-api" containerID="cri-o://db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e" gracePeriod=30
Jul 27 17:53:04.869361 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:04.869323    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:04.970464 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:04.970429    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:05.026120 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.026079    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:05.103679 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.103614    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:53:05.162106 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.162067    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8e992b-b37b-4416-aa2a-f182c0701914-maas-api-tls\") pod \"cc8e992b-b37b-4416-aa2a-f182c0701914\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") "
Jul 27 17:53:05.162106 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.162108    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8e992b-b37b-4416-aa2a-f182c0701914-openshift-service-ca\") pod \"cc8e992b-b37b-4416-aa2a-f182c0701914\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") "
Jul 27 17:53:05.162339 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.162146    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpjwr\" (UniqueName: \"kubernetes.io/projected/cc8e992b-b37b-4416-aa2a-f182c0701914-kube-api-access-rpjwr\") pod \"cc8e992b-b37b-4416-aa2a-f182c0701914\" (UID: \"cc8e992b-b37b-4416-aa2a-f182c0701914\") "
Jul 27 17:53:05.162531 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.162505    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc8e992b-b37b-4416-aa2a-f182c0701914-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc8e992b-b37b-4416-aa2a-f182c0701914" (UID: "cc8e992b-b37b-4416-aa2a-f182c0701914"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:53:05.164493 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.164465    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc8e992b-b37b-4416-aa2a-f182c0701914-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc8e992b-b37b-4416-aa2a-f182c0701914" (UID: "cc8e992b-b37b-4416-aa2a-f182c0701914"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:53:05.164584 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.164493    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc8e992b-b37b-4416-aa2a-f182c0701914-kube-api-access-rpjwr" (OuterVolumeSpecName: "kube-api-access-rpjwr") pod "cc8e992b-b37b-4416-aa2a-f182c0701914" (UID: "cc8e992b-b37b-4416-aa2a-f182c0701914"). InnerVolumeSpecName "kube-api-access-rpjwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:53:05.263006 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.262963    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8e992b-b37b-4416-aa2a-f182c0701914-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:53:05.263006 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.262999    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8e992b-b37b-4416-aa2a-f182c0701914-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:53:05.263006 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.263012    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpjwr\" (UniqueName: \"kubernetes.io/projected/cc8e992b-b37b-4416-aa2a-f182c0701914-kube-api-access-rpjwr\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:53:05.423226 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.423140    2584 generic.go:356] "Generic (PLEG): container finished" podID="cc8e992b-b37b-4416-aa2a-f182c0701914" containerID="db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e" exitCode=0
Jul 27 17:53:05.423226 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.423201    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"
Jul 27 17:53:05.423226 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.423211    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p" event={"ID":"cc8e992b-b37b-4416-aa2a-f182c0701914","Type":"ContainerDied","Data":"db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e"}
Jul 27 17:53:05.423490 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.423252    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p" event={"ID":"cc8e992b-b37b-4416-aa2a-f182c0701914","Type":"ContainerDied","Data":"9275f7d9153ebf814803368e0b83ea2f4e0234bb1fe444c02e8d89218bb998a3"}
Jul 27 17:53:05.423490 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.423283    2584 scope.go:117] "RemoveContainer" containerID="db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e"
Jul 27 17:53:05.435189 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.435160    2584 scope.go:117] "RemoveContainer" containerID="db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e"
Jul 27 17:53:05.435786 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:53:05.435762    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e\": container with ID starting with db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e not found: ID does not exist" containerID="db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e"
Jul 27 17:53:05.435928 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.435797    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e"} err="failed to get container status \"db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e\": rpc error: code = NotFound desc = could not find container \"db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e\": container with ID starting with db6c785726b036623143e7f781b849a12f53f73fa4766983750bc001082f439e not found: ID does not exist"
Jul 27 17:53:05.446132 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.446105    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"]
Jul 27 17:53:05.450148 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:05.450118    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3852632a-859fff7c74-ngd4p"]
Jul 27 17:53:06.823110 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:06.823074    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8e992b-b37b-4416-aa2a-f182c0701914" path="/var/lib/kubelet/pods/cc8e992b-b37b-4416-aa2a-f182c0701914/volumes"
Jul 27 17:53:09.477983 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:09.477948    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:09.865996 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:09.865961    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:16.045763 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:16.045720    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:16.138633 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:16.138578    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:16.364246 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:16.364205    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:16.550440 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:16.550369    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:16.691066 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:16.690946    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:16.797564 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:16.797529    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:17.029910 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:17.029789    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:22.963424 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:22.963372    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:23.067360 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:23.067306    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:23.167277 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:23.167243    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:23.331193 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:23.331159    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:29.022743 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:29.022672    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:29.086538 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:29.086500    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:33.298036 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:33.297999    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:35.866999 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:35.866958    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:36.190254 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:36.190151    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:36.374653 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:36.374618    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:38.856781 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:38.856734    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:38.940008 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:38.939960    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:38.941857 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:38.941826    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"]
Jul 27 17:53:38.942263 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:38.942249    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8e992b-b37b-4416-aa2a-f182c0701914" containerName="maas-api"
Jul 27 17:53:38.942314 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:38.942265    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8e992b-b37b-4416-aa2a-f182c0701914" containerName="maas-api"
Jul 27 17:53:38.942350 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:38.942329    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8e992b-b37b-4416-aa2a-f182c0701914" containerName="maas-api"
Jul 27 17:53:38.942350 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:38.942340    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bed79d33-ce8a-4f12-b0d8-26dfc3f0a658" containerName="istio-proxy"
Jul 27 17:53:38.945450 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:38.945431    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:38.948670 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:38.948651    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1b693e0c-serving-cert\""
Jul 27 17:53:38.956541 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:38.956513    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"]
Jul 27 17:53:39.001211 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.001163    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7476b\" (UniqueName: \"kubernetes.io/projected/e3ae876b-5791-4a8b-8266-fd360aecfc4f-kube-api-access-7476b\") pod \"maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:39.001415 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.001235    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ae876b-5791-4a8b-8266-fd360aecfc4f-maas-api-tls\") pod \"maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:39.001415 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.001323    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ae876b-5791-4a8b-8266-fd360aecfc4f-openshift-service-ca\") pod \"maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:39.012461 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.012412    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:39.102801 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.102764    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7476b\" (UniqueName: \"kubernetes.io/projected/e3ae876b-5791-4a8b-8266-fd360aecfc4f-kube-api-access-7476b\") pod \"maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:39.103013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.102813    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ae876b-5791-4a8b-8266-fd360aecfc4f-maas-api-tls\") pod \"maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:39.103013 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.102864    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ae876b-5791-4a8b-8266-fd360aecfc4f-openshift-service-ca\") pod \"maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:39.103013 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:53:39.102983    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-serving-cert: secret "maas-api-e2e-mt-1b693e0c-serving-cert" not found
Jul 27 17:53:39.103179 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:53:39.103060    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3ae876b-5791-4a8b-8266-fd360aecfc4f-maas-api-tls podName:e3ae876b-5791-4a8b-8266-fd360aecfc4f nodeName:}" failed. No retries permitted until 2026-07-27 17:53:39.603037887 +0000 UTC m=+1957.311186183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e3ae876b-5791-4a8b-8266-fd360aecfc4f-maas-api-tls") pod "maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh" (UID: "e3ae876b-5791-4a8b-8266-fd360aecfc4f") : secret "maas-api-e2e-mt-1b693e0c-serving-cert" not found
Jul 27 17:53:39.103597 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.103576    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ae876b-5791-4a8b-8266-fd360aecfc4f-openshift-service-ca\") pod \"maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:39.115641 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.115569    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7476b\" (UniqueName: \"kubernetes.io/projected/e3ae876b-5791-4a8b-8266-fd360aecfc4f-kube-api-access-7476b\") pod \"maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:39.226895 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.226859    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:39.608660 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.608624    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ae876b-5791-4a8b-8266-fd360aecfc4f-maas-api-tls\") pod \"maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:39.611383 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.611359    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ae876b-5791-4a8b-8266-fd360aecfc4f-maas-api-tls\") pod \"maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:39.656484 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.656442    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:39.753210 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.753158    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:39.858128 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:39.858093    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:40.020864 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:40.020788    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"]
Jul 27 17:53:40.023226 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:53:40.023201    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3ae876b_5791_4a8b_8266_fd360aecfc4f.slice/crio-a15042449132075d11087240759a464ae53e0f231c2fb3fc1c6decab9274edb6 WatchSource:0}: Error finding container a15042449132075d11087240759a464ae53e0f231c2fb3fc1c6decab9274edb6: Status 404 returned error can't find the container with id a15042449132075d11087240759a464ae53e0f231c2fb3fc1c6decab9274edb6
Jul 27 17:53:40.024563 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:40.024545    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:53:40.602729 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:40.602663    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh" event={"ID":"e3ae876b-5791-4a8b-8266-fd360aecfc4f","Type":"ContainerStarted","Data":"a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8"}
Jul 27 17:53:40.602944 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:40.602767    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:40.602944 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:40.602793    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh" event={"ID":"e3ae876b-5791-4a8b-8266-fd360aecfc4f","Type":"ContainerStarted","Data":"a15042449132075d11087240759a464ae53e0f231c2fb3fc1c6decab9274edb6"}
Jul 27 17:53:40.622903 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:40.622851    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh" podStartSLOduration=2.183517497 podStartE2EDuration="2.622835252s" totalImagesPullingTime="439.317755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:53:40.620740522 +0000 UTC m=+1958.328888842" watchObservedRunningTime="2026-07-27 17:53:40.622835252 +0000 UTC m=+1958.330983569"
Jul 27 17:53:43.372172 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:43.372136    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:46.613469 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:46.613433    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:53:56.286635 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:56.286537    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:59.099537 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:53:59.099496    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:09.449348 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:54:09.449301    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:04.055011 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.054963    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"]
Jul 27 17:55:04.055637 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.055243    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh" podUID="e3ae876b-5791-4a8b-8266-fd360aecfc4f" containerName="maas-api" containerID="cri-o://a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8" gracePeriod=30
Jul 27 17:55:04.077606 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.077576    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:04.145451 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.145413    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:04.208433 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.208401    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:04.313403 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.313380    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:55:04.474896 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.474858    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ae876b-5791-4a8b-8266-fd360aecfc4f-maas-api-tls\") pod \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") "
Jul 27 17:55:04.475090 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.474926    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ae876b-5791-4a8b-8266-fd360aecfc4f-openshift-service-ca\") pod \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") "
Jul 27 17:55:04.475090 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.474970    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7476b\" (UniqueName: \"kubernetes.io/projected/e3ae876b-5791-4a8b-8266-fd360aecfc4f-kube-api-access-7476b\") pod \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\" (UID: \"e3ae876b-5791-4a8b-8266-fd360aecfc4f\") "
Jul 27 17:55:04.475349 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.475323    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3ae876b-5791-4a8b-8266-fd360aecfc4f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e3ae876b-5791-4a8b-8266-fd360aecfc4f" (UID: "e3ae876b-5791-4a8b-8266-fd360aecfc4f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:55:04.477285 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.477263    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3ae876b-5791-4a8b-8266-fd360aecfc4f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e3ae876b-5791-4a8b-8266-fd360aecfc4f" (UID: "e3ae876b-5791-4a8b-8266-fd360aecfc4f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:55:04.477381 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.477327    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3ae876b-5791-4a8b-8266-fd360aecfc4f-kube-api-access-7476b" (OuterVolumeSpecName: "kube-api-access-7476b") pod "e3ae876b-5791-4a8b-8266-fd360aecfc4f" (UID: "e3ae876b-5791-4a8b-8266-fd360aecfc4f"). InnerVolumeSpecName "kube-api-access-7476b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:55:04.576045 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.575953    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7476b\" (UniqueName: \"kubernetes.io/projected/e3ae876b-5791-4a8b-8266-fd360aecfc4f-kube-api-access-7476b\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:55:04.576045 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.575985    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ae876b-5791-4a8b-8266-fd360aecfc4f-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:55:04.576045 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:04.575995    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ae876b-5791-4a8b-8266-fd360aecfc4f-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:55:05.016934 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:05.016896    2584 generic.go:356] "Generic (PLEG): container finished" podID="e3ae876b-5791-4a8b-8266-fd360aecfc4f" containerID="a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8" exitCode=0
Jul 27 17:55:05.017138 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:05.016953    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"
Jul 27 17:55:05.017138 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:05.016982    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh" event={"ID":"e3ae876b-5791-4a8b-8266-fd360aecfc4f","Type":"ContainerDied","Data":"a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8"}
Jul 27 17:55:05.017138 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:05.017035    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh" event={"ID":"e3ae876b-5791-4a8b-8266-fd360aecfc4f","Type":"ContainerDied","Data":"a15042449132075d11087240759a464ae53e0f231c2fb3fc1c6decab9274edb6"}
Jul 27 17:55:05.017138 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:05.017057    2584 scope.go:117] "RemoveContainer" containerID="a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8"
Jul 27 17:55:05.026286 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:05.026267    2584 scope.go:117] "RemoveContainer" containerID="a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8"
Jul 27 17:55:05.026561 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:55:05.026538    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8\": container with ID starting with a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8 not found: ID does not exist" containerID="a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8"
Jul 27 17:55:05.026620 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:05.026571    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8"} err="failed to get container status \"a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8\": rpc error: code = NotFound desc = could not find container \"a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8\": container with ID starting with a414efb0d5429bab6120de6bae8bb047194f9516fe258741eaf9f713ad17bcd8 not found: ID does not exist"
Jul 27 17:55:05.034302 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:05.034276    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"]
Jul 27 17:55:05.037577 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:05.037548    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b693e0c-f789c65dc-rcggh"]
Jul 27 17:55:06.824213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:06.824179    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3ae876b-5791-4a8b-8266-fd360aecfc4f" path="/var/lib/kubelet/pods/e3ae876b-5791-4a8b-8266-fd360aecfc4f/volumes"
Jul 27 17:55:08.668255 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:08.668220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:11.875988 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:11.875944    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:16.956453 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:16.956417    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:53.619179 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:53.619142    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:54.036293 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:54.036196    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:59.547863 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:55:59.547826    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:22.535906 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:22.535863    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:22.792462 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:22.792352    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:22.860142 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:22.860103    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:23.082359 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:23.082328    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:24.132276 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:24.132226    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:24.231823 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:24.231787    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:24.331501 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:24.331462    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:25.054312 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:25.054274    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:25.156784 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:25.156738    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:25.327751 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:25.327712    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:26.723908 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:26.723873    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:32.337221 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:32.337186    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:32.724139 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:32.724041    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:32.835821 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:32.835783    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:33.455840 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:33.455802    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:39.305628 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:39.305581    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:39.415315 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:39.415275    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:39.506256 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:39.506212    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:50.031599 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:50.031510    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:50.379050 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:56:50.379012    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:11.369347 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:11.369257    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:17.567323 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:17.567286    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:17.676180 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:17.676143    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:17.938307 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:17.938205    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:18.471971 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:18.471905    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:18.522965 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:18.522933    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:21.105223 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.105179    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:21.186423 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.186388    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"]
Jul 27 17:57:21.186938 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.186915    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3ae876b-5791-4a8b-8266-fd360aecfc4f" containerName="maas-api"
Jul 27 17:57:21.186938 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.186935    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3ae876b-5791-4a8b-8266-fd360aecfc4f" containerName="maas-api"
Jul 27 17:57:21.187107 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.187012    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3ae876b-5791-4a8b-8266-fd360aecfc4f" containerName="maas-api"
Jul 27 17:57:21.189891 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.189872    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.192698 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.192660    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a155ed-serving-cert\""
Jul 27 17:57:21.201810 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.201783    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"]
Jul 27 17:57:21.345134 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.345093    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a529b2eb-7892-4046-bc7c-eca4c26701d6-maas-api-tls\") pod \"maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.345134 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.345129    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a529b2eb-7892-4046-bc7c-eca4c26701d6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.345410 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.345168    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvlxm\" (UniqueName: \"kubernetes.io/projected/a529b2eb-7892-4046-bc7c-eca4c26701d6-kube-api-access-kvlxm\") pod \"maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.446229 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.446113    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a529b2eb-7892-4046-bc7c-eca4c26701d6-maas-api-tls\") pod \"maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.446229 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.446158    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a529b2eb-7892-4046-bc7c-eca4c26701d6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.446229 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.446195    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvlxm\" (UniqueName: \"kubernetes.io/projected/a529b2eb-7892-4046-bc7c-eca4c26701d6-kube-api-access-kvlxm\") pod \"maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.446531 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:57:21.446300    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-serving-cert: secret "maas-api-e2e-inf-a-a155ed-serving-cert" not found
Jul 27 17:57:21.446531 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:57:21.446404    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a529b2eb-7892-4046-bc7c-eca4c26701d6-maas-api-tls podName:a529b2eb-7892-4046-bc7c-eca4c26701d6 nodeName:}" failed. No retries permitted until 2026-07-27 17:57:21.946378962 +0000 UTC m=+2179.654527259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a529b2eb-7892-4046-bc7c-eca4c26701d6-maas-api-tls") pod "maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h" (UID: "a529b2eb-7892-4046-bc7c-eca4c26701d6") : secret "maas-api-e2e-inf-a-a155ed-serving-cert" not found
Jul 27 17:57:21.446883 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.446862    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a529b2eb-7892-4046-bc7c-eca4c26701d6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.455318 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.455294    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvlxm\" (UniqueName: \"kubernetes.io/projected/a529b2eb-7892-4046-bc7c-eca4c26701d6-kube-api-access-kvlxm\") pod \"maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.600927 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.600884    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:21.697286 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.697189    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:21.890621 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.890579    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:21.952176 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.952085    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a529b2eb-7892-4046-bc7c-eca4c26701d6-maas-api-tls\") pod \"maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.954722 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.954679    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a529b2eb-7892-4046-bc7c-eca4c26701d6-maas-api-tls\") pod \"maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:21.993771 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:21.993738    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:22.034825 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.034790    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"]
Jul 27 17:57:22.038451 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.038425    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:57:22.041519 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.041477    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-a155ed-dockercfg-hg8vh\""
Jul 27 17:57:22.041519 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.041477    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-a155ed\""
Jul 27 17:57:22.046092 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.046063    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"]
Jul 27 17:57:22.102982 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.102948    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:22.154193 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.154150    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a34d56-fc58-4a51-9f68-fbc677ca2e04-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg\" (UID: \"14a34d56-fc58-4a51-9f68-fbc677ca2e04\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:57:22.154670 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.154286    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8rrv\" (UniqueName: \"kubernetes.io/projected/14a34d56-fc58-4a51-9f68-fbc677ca2e04-kube-api-access-p8rrv\") pod \"payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg\" (UID: \"14a34d56-fc58-4a51-9f68-fbc677ca2e04\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:57:22.246218 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.246191    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"]
Jul 27 17:57:22.248475 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:57:22.248448    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda529b2eb_7892_4046_bc7c_eca4c26701d6.slice/crio-cb5a81640a928e26a01a571e1512380983c9eeeb7303cf6cc80095d4eaa27211 WatchSource:0}: Error finding container cb5a81640a928e26a01a571e1512380983c9eeeb7303cf6cc80095d4eaa27211: Status 404 returned error can't find the container with id cb5a81640a928e26a01a571e1512380983c9eeeb7303cf6cc80095d4eaa27211
Jul 27 17:57:22.254988 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.254923    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8rrv\" (UniqueName: \"kubernetes.io/projected/14a34d56-fc58-4a51-9f68-fbc677ca2e04-kube-api-access-p8rrv\") pod \"payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg\" (UID: \"14a34d56-fc58-4a51-9f68-fbc677ca2e04\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:57:22.255136 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.255094    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a34d56-fc58-4a51-9f68-fbc677ca2e04-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg\" (UID: \"14a34d56-fc58-4a51-9f68-fbc677ca2e04\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:57:22.255955 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.255933    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a34d56-fc58-4a51-9f68-fbc677ca2e04-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg\" (UID: \"14a34d56-fc58-4a51-9f68-fbc677ca2e04\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:57:22.263270 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.263248    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8rrv\" (UniqueName: \"kubernetes.io/projected/14a34d56-fc58-4a51-9f68-fbc677ca2e04-kube-api-access-p8rrv\") pod \"payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg\" (UID: \"14a34d56-fc58-4a51-9f68-fbc677ca2e04\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:57:22.352944 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.352906    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:57:22.493998 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.493971    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"]
Jul 27 17:57:22.496207 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:57:22.496140    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a34d56_fc58_4a51_9f68_fbc677ca2e04.slice/crio-870635411f0812d92de361e8993444db739d0cf4f4b5b5dec796371ff83bd4ec WatchSource:0}: Error finding container 870635411f0812d92de361e8993444db739d0cf4f4b5b5dec796371ff83bd4ec: Status 404 returned error can't find the container with id 870635411f0812d92de361e8993444db739d0cf4f4b5b5dec796371ff83bd4ec
Jul 27 17:57:22.733574 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.733538    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h" event={"ID":"a529b2eb-7892-4046-bc7c-eca4c26701d6","Type":"ContainerStarted","Data":"cb5a81640a928e26a01a571e1512380983c9eeeb7303cf6cc80095d4eaa27211"}
Jul 27 17:57:22.735005 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.734980    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg" event={"ID":"14a34d56-fc58-4a51-9f68-fbc677ca2e04","Type":"ContainerStarted","Data":"cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d"}
Jul 27 17:57:22.735124 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.735021    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg" event={"ID":"14a34d56-fc58-4a51-9f68-fbc677ca2e04","Type":"ContainerStarted","Data":"870635411f0812d92de361e8993444db739d0cf4f4b5b5dec796371ff83bd4ec"}
Jul 27 17:57:22.735124 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.735094    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:57:22.755309 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:22.755202    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg" podStartSLOduration=0.755184938 podStartE2EDuration="755.184938ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:57:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:57:22.751863561 +0000 UTC m=+2180.460011881" watchObservedRunningTime="2026-07-27 17:57:22.755184938 +0000 UTC m=+2180.463333256"
Jul 27 17:57:23.742553 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:23.742518    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h" event={"ID":"a529b2eb-7892-4046-bc7c-eca4c26701d6","Type":"ContainerStarted","Data":"073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669"}
Jul 27 17:57:23.743043 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:23.742820    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:23.761853 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:23.761797    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h" podStartSLOduration=2.200488051 podStartE2EDuration="2.76177629s" totalImagesPullingTime="561.288239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:57:23.76088154 +0000 UTC m=+2181.469029868" watchObservedRunningTime="2026-07-27 17:57:23.76177629 +0000 UTC m=+2181.469924610"
Jul 27 17:57:25.632468 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:25.632410    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:26.213720 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:26.213658    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:26.306975 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:26.306937    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:26.381936 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:26.381903    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:29.128828 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.128788    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:29.231840 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.231794    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:29.490027 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.489950    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"]
Jul 27 17:57:29.493866 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.493846    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.497081 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.497058    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ab2501-serving-cert\""
Jul 27 17:57:29.505046 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.505019    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"]
Jul 27 17:57:29.521453 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.521423    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:29.621155 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.621033    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:29.627349 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.627314    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.627515 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.627376    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.627559 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.627541    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgdf2\" (UniqueName: \"kubernetes.io/projected/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-kube-api-access-cgdf2\") pod \"maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.729213 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.729172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.729373 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.729234    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.729373 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.729351    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgdf2\" (UniqueName: \"kubernetes.io/projected/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-kube-api-access-cgdf2\") pod \"maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.729960 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.729931    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.731817 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.731791    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.741673 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.741598    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgdf2\" (UniqueName: \"kubernetes.io/projected/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-kube-api-access-cgdf2\") pod \"maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.754620 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.754586    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:57:29.807398 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.807363    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:29.958224 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:29.958200    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"]
Jul 27 17:57:29.963649 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:57:29.963610    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d416fcb_4ac4_42d1_af5d_5f8e5b9d2e3f.slice/crio-bc3d4f2d1a5f76c31484837a28fc7ce33c1023c7257bc9298cdea84f2499eb56 WatchSource:0}: Error finding container bc3d4f2d1a5f76c31484837a28fc7ce33c1023c7257bc9298cdea84f2499eb56: Status 404 returned error can't find the container with id bc3d4f2d1a5f76c31484837a28fc7ce33c1023c7257bc9298cdea84f2499eb56
Jul 27 17:57:30.150632 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.150590    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:30.249743 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.249673    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:30.331712 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.331651    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"]
Jul 27 17:57:30.336434 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.336408    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:57:30.339393 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.339181    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ab2501-dockercfg-gcp4c\""
Jul 27 17:57:30.339393 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.339216    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ab2501\""
Jul 27 17:57:30.343095 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.343067    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"]
Jul 27 17:57:30.437556 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.437457    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6526ad-54e8-448f-bd4f-204aaf53e6db-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk\" (UID: \"5d6526ad-54e8-448f-bd4f-204aaf53e6db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:57:30.437747 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.437641    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5mbc\" (UniqueName: \"kubernetes.io/projected/5d6526ad-54e8-448f-bd4f-204aaf53e6db-kube-api-access-l5mbc\") pod \"payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk\" (UID: \"5d6526ad-54e8-448f-bd4f-204aaf53e6db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:57:30.539127 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.539089    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5mbc\" (UniqueName: \"kubernetes.io/projected/5d6526ad-54e8-448f-bd4f-204aaf53e6db-kube-api-access-l5mbc\") pod \"payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk\" (UID: \"5d6526ad-54e8-448f-bd4f-204aaf53e6db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:57:30.539127 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.539134    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6526ad-54e8-448f-bd4f-204aaf53e6db-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk\" (UID: \"5d6526ad-54e8-448f-bd4f-204aaf53e6db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:57:30.539905 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.539885    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6526ad-54e8-448f-bd4f-204aaf53e6db-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk\" (UID: \"5d6526ad-54e8-448f-bd4f-204aaf53e6db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:57:30.553791 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.553768    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5mbc\" (UniqueName: \"kubernetes.io/projected/5d6526ad-54e8-448f-bd4f-204aaf53e6db-kube-api-access-l5mbc\") pod \"payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk\" (UID: \"5d6526ad-54e8-448f-bd4f-204aaf53e6db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:57:30.651481 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.651452    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:57:30.781984 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.781939    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b" event={"ID":"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f","Type":"ContainerStarted","Data":"ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7"}
Jul 27 17:57:30.782168 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.782007    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:30.782168 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.782033    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b" event={"ID":"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f","Type":"ContainerStarted","Data":"bc3d4f2d1a5f76c31484837a28fc7ce33c1023c7257bc9298cdea84f2499eb56"}
Jul 27 17:57:30.796921 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.796896    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"]
Jul 27 17:57:30.799396 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:57:30.799361    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d6526ad_54e8_448f_bd4f_204aaf53e6db.slice/crio-ebec2e7d880fbf3f2e79c49c21c02330b508888272bfe286b894d60166cb6a52 WatchSource:0}: Error finding container ebec2e7d880fbf3f2e79c49c21c02330b508888272bfe286b894d60166cb6a52: Status 404 returned error can't find the container with id ebec2e7d880fbf3f2e79c49c21c02330b508888272bfe286b894d60166cb6a52
Jul 27 17:57:30.799945 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:30.799797    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b" podStartSLOduration=1.165277481 podStartE2EDuration="1.799779186s" totalImagesPullingTime="634.501705ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:57:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:57:30.799605627 +0000 UTC m=+2188.507753946" watchObservedRunningTime="2026-07-27 17:57:30.799779186 +0000 UTC m=+2188.507927505"
Jul 27 17:57:31.788592 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:31.788556    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk" event={"ID":"5d6526ad-54e8-448f-bd4f-204aaf53e6db","Type":"ContainerStarted","Data":"3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1"}
Jul 27 17:57:31.789004 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:31.788614    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk" event={"ID":"5d6526ad-54e8-448f-bd4f-204aaf53e6db","Type":"ContainerStarted","Data":"ebec2e7d880fbf3f2e79c49c21c02330b508888272bfe286b894d60166cb6a52"}
Jul 27 17:57:31.806623 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:31.806573    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk" podStartSLOduration=1.8065560330000001 podStartE2EDuration="1.806556033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:57:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:57:31.804648005 +0000 UTC m=+2189.512796324" watchObservedRunningTime="2026-07-27 17:57:31.806556033 +0000 UTC m=+2189.514704351"
Jul 27 17:57:32.794302 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:32.794266    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:57:33.680723 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:33.680665    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:33.745407 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:33.745381    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:57:36.793870 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:36.793839    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:57:43.801189 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:43.801154    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:57:52.181869 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:57:52.181827    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:14.297598 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:58:14.297544    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:53.398096 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:58:53.398058    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:54.387347 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:58:54.387302    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:56.508422 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:58:56.508389    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:02.980347 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:02.980302    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:03.035174 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:03.035140    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:04.906883 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:04.906848    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"]
Jul 27 17:59:04.907367 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:04.907093    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h" podUID="a529b2eb-7892-4046-bc7c-eca4c26701d6" containerName="maas-api" containerID="cri-o://073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669" gracePeriod=30
Jul 27 17:59:04.927895 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:04.927868    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:04.962011 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:04.961974    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"]
Jul 27 17:59:04.962225 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:04.962204    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg" podUID="14a34d56-fc58-4a51-9f68-fbc677ca2e04" containerName="payload-processing" containerID="cri-o://cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d" gracePeriod=30
Jul 27 17:59:04.995709 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:04.995638    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:05.063505 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.063474    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:05.239259 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.239227    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:59:05.274264 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.274237    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:59:05.276177 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.276152    2584 generic.go:356] "Generic (PLEG): container finished" podID="a529b2eb-7892-4046-bc7c-eca4c26701d6" containerID="073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669" exitCode=0
Jul 27 17:59:05.276301 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.276202    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"
Jul 27 17:59:05.276301 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.276237    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h" event={"ID":"a529b2eb-7892-4046-bc7c-eca4c26701d6","Type":"ContainerDied","Data":"073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669"}
Jul 27 17:59:05.276301 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.276291    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h" event={"ID":"a529b2eb-7892-4046-bc7c-eca4c26701d6","Type":"ContainerDied","Data":"cb5a81640a928e26a01a571e1512380983c9eeeb7303cf6cc80095d4eaa27211"}
Jul 27 17:59:05.276458 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.276317    2584 scope.go:117] "RemoveContainer" containerID="073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669"
Jul 27 17:59:05.277961 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.277938    2584 generic.go:356] "Generic (PLEG): container finished" podID="14a34d56-fc58-4a51-9f68-fbc677ca2e04" containerID="cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d" exitCode=0
Jul 27 17:59:05.278083 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.277985    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg" event={"ID":"14a34d56-fc58-4a51-9f68-fbc677ca2e04","Type":"ContainerDied","Data":"cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d"}
Jul 27 17:59:05.278083 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.277994    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"
Jul 27 17:59:05.278083 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.278029    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg" event={"ID":"14a34d56-fc58-4a51-9f68-fbc677ca2e04","Type":"ContainerDied","Data":"870635411f0812d92de361e8993444db739d0cf4f4b5b5dec796371ff83bd4ec"}
Jul 27 17:59:05.285361 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.285341    2584 scope.go:117] "RemoveContainer" containerID="073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669"
Jul 27 17:59:05.285721 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:59:05.285680    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669\": container with ID starting with 073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669 not found: ID does not exist" containerID="073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669"
Jul 27 17:59:05.285806 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.285735    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669"} err="failed to get container status \"073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669\": rpc error: code = NotFound desc = could not find container \"073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669\": container with ID starting with 073d9522ac371b0b156a55ae60f78417712124df67f704f65ca7030deb30f669 not found: ID does not exist"
Jul 27 17:59:05.285806 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.285761    2584 scope.go:117] "RemoveContainer" containerID="cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d"
Jul 27 17:59:05.297357 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.297334    2584 scope.go:117] "RemoveContainer" containerID="cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d"
Jul 27 17:59:05.297743 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:59:05.297715    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d\": container with ID starting with cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d not found: ID does not exist" containerID="cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d"
Jul 27 17:59:05.297866 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.297751    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d"} err="failed to get container status \"cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d\": rpc error: code = NotFound desc = could not find container \"cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d\": container with ID starting with cc7fe1f9f22edb5dc85885410c4a339f947e9e2a5dc309415065a4dffb6b2b6d not found: ID does not exist"
Jul 27 17:59:05.399182 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.399142    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a529b2eb-7892-4046-bc7c-eca4c26701d6-openshift-service-ca\") pod \"a529b2eb-7892-4046-bc7c-eca4c26701d6\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") "
Jul 27 17:59:05.399182 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.399194    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a529b2eb-7892-4046-bc7c-eca4c26701d6-maas-api-tls\") pod \"a529b2eb-7892-4046-bc7c-eca4c26701d6\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") "
Jul 27 17:59:05.399463 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.399249    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvlxm\" (UniqueName: \"kubernetes.io/projected/a529b2eb-7892-4046-bc7c-eca4c26701d6-kube-api-access-kvlxm\") pod \"a529b2eb-7892-4046-bc7c-eca4c26701d6\" (UID: \"a529b2eb-7892-4046-bc7c-eca4c26701d6\") "
Jul 27 17:59:05.399463 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.399302    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8rrv\" (UniqueName: \"kubernetes.io/projected/14a34d56-fc58-4a51-9f68-fbc677ca2e04-kube-api-access-p8rrv\") pod \"14a34d56-fc58-4a51-9f68-fbc677ca2e04\" (UID: \"14a34d56-fc58-4a51-9f68-fbc677ca2e04\") "
Jul 27 17:59:05.399463 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.399349    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a34d56-fc58-4a51-9f68-fbc677ca2e04-plugins-config-volume\") pod \"14a34d56-fc58-4a51-9f68-fbc677ca2e04\" (UID: \"14a34d56-fc58-4a51-9f68-fbc677ca2e04\") "
Jul 27 17:59:05.399627 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.399604    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a529b2eb-7892-4046-bc7c-eca4c26701d6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a529b2eb-7892-4046-bc7c-eca4c26701d6" (UID: "a529b2eb-7892-4046-bc7c-eca4c26701d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:59:05.399807 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.399788    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a529b2eb-7892-4046-bc7c-eca4c26701d6-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:59:05.399881 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.399821    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14a34d56-fc58-4a51-9f68-fbc677ca2e04-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "14a34d56-fc58-4a51-9f68-fbc677ca2e04" (UID: "14a34d56-fc58-4a51-9f68-fbc677ca2e04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:59:05.401707 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.401662    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a529b2eb-7892-4046-bc7c-eca4c26701d6-kube-api-access-kvlxm" (OuterVolumeSpecName: "kube-api-access-kvlxm") pod "a529b2eb-7892-4046-bc7c-eca4c26701d6" (UID: "a529b2eb-7892-4046-bc7c-eca4c26701d6"). InnerVolumeSpecName "kube-api-access-kvlxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:59:05.401812 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.401678    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14a34d56-fc58-4a51-9f68-fbc677ca2e04-kube-api-access-p8rrv" (OuterVolumeSpecName: "kube-api-access-p8rrv") pod "14a34d56-fc58-4a51-9f68-fbc677ca2e04" (UID: "14a34d56-fc58-4a51-9f68-fbc677ca2e04"). InnerVolumeSpecName "kube-api-access-p8rrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:59:05.401812 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.401676    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a529b2eb-7892-4046-bc7c-eca4c26701d6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a529b2eb-7892-4046-bc7c-eca4c26701d6" (UID: "a529b2eb-7892-4046-bc7c-eca4c26701d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:59:05.501225 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.501117    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvlxm\" (UniqueName: \"kubernetes.io/projected/a529b2eb-7892-4046-bc7c-eca4c26701d6-kube-api-access-kvlxm\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:59:05.501225 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.501154    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8rrv\" (UniqueName: \"kubernetes.io/projected/14a34d56-fc58-4a51-9f68-fbc677ca2e04-kube-api-access-p8rrv\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:59:05.501225 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.501169    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a34d56-fc58-4a51-9f68-fbc677ca2e04-plugins-config-volume\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:59:05.501225 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.501183    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a529b2eb-7892-4046-bc7c-eca4c26701d6-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:59:05.600904 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.600870    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"]
Jul 27 17:59:05.604769 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.604731    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a155ed-5c5cc46c4f-b7x6h"]
Jul 27 17:59:05.616772 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.616742    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"]
Jul 27 17:59:05.620357 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:05.620331    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a155ed-65fbbf8864-kqgpg"]
Jul 27 17:59:06.823001 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:06.822964    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14a34d56-fc58-4a51-9f68-fbc677ca2e04" path="/var/lib/kubelet/pods/14a34d56-fc58-4a51-9f68-fbc677ca2e04/volumes"
Jul 27 17:59:06.823573 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:06.823550    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a529b2eb-7892-4046-bc7c-eca4c26701d6" path="/var/lib/kubelet/pods/a529b2eb-7892-4046-bc7c-eca4c26701d6/volumes"
Jul 27 17:59:09.567923 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:09.567889    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:09.976093 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:09.975995    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:15.886483 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:15.886451    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:15.947243 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:15.947202    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:16.143520 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:16.143423    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:16.373707 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:16.373658    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:26.955054 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:26.955019    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"]
Jul 27 17:59:26.955585 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:26.955228    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b" podUID="3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f" containerName="maas-api" containerID="cri-o://ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7" gracePeriod=30
Jul 27 17:59:26.977752 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:26.977722    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:27.012298 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.012264    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"]
Jul 27 17:59:27.012593 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.012562    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk" podUID="5d6526ad-54e8-448f-bd4f-204aaf53e6db" containerName="payload-pre-processing" containerID="cri-o://3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1" gracePeriod=30
Jul 27 17:59:27.044734 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.044678    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:27.147408 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.147353    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:27.271174 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.271146    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:59:27.299876 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.299845    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:59:27.319958 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.319929    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-openshift-service-ca\") pod \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") "
Jul 27 17:59:27.320156 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.319996    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-maas-api-tls\") pod \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") "
Jul 27 17:59:27.320156 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.320032    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgdf2\" (UniqueName: \"kubernetes.io/projected/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-kube-api-access-cgdf2\") pod \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\" (UID: \"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f\") "
Jul 27 17:59:27.320367 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.320339    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f" (UID: "3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:59:27.322409 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.322374    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-kube-api-access-cgdf2" (OuterVolumeSpecName: "kube-api-access-cgdf2") pod "3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f" (UID: "3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f"). InnerVolumeSpecName "kube-api-access-cgdf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:59:27.322530 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.322438    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f" (UID: "3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:59:27.394482 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.394441    2584 generic.go:356] "Generic (PLEG): container finished" podID="5d6526ad-54e8-448f-bd4f-204aaf53e6db" containerID="3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1" exitCode=0
Jul 27 17:59:27.394661 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.394493    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"
Jul 27 17:59:27.394661 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.394530    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk" event={"ID":"5d6526ad-54e8-448f-bd4f-204aaf53e6db","Type":"ContainerDied","Data":"3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1"}
Jul 27 17:59:27.394661 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.394586    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk" event={"ID":"5d6526ad-54e8-448f-bd4f-204aaf53e6db","Type":"ContainerDied","Data":"ebec2e7d880fbf3f2e79c49c21c02330b508888272bfe286b894d60166cb6a52"}
Jul 27 17:59:27.394661 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.394616    2584 scope.go:117] "RemoveContainer" containerID="3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1"
Jul 27 17:59:27.395811 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.395797    2584 generic.go:356] "Generic (PLEG): container finished" podID="3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f" containerID="ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7" exitCode=0
Jul 27 17:59:27.395877 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.395844    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"
Jul 27 17:59:27.395918 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.395889    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b" event={"ID":"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f","Type":"ContainerDied","Data":"ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7"}
Jul 27 17:59:27.395963 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.395934    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b" event={"ID":"3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f","Type":"ContainerDied","Data":"bc3d4f2d1a5f76c31484837a28fc7ce33c1023c7257bc9298cdea84f2499eb56"}
Jul 27 17:59:27.404703 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.404664    2584 scope.go:117] "RemoveContainer" containerID="3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1"
Jul 27 17:59:27.405008 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:59:27.404988    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1\": container with ID starting with 3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1 not found: ID does not exist" containerID="3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1"
Jul 27 17:59:27.405097 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.405016    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1"} err="failed to get container status \"3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1\": rpc error: code = NotFound desc = could not find container \"3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1\": container with ID starting with 3dae33f3c614e693aa1d0b73923fbfd70bcae71d80e20b954e5160b9e51192d1 not found: ID does not exist"
Jul 27 17:59:27.405097 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.405031    2584 scope.go:117] "RemoveContainer" containerID="ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7"
Jul 27 17:59:27.413807 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.413789    2584 scope.go:117] "RemoveContainer" containerID="ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7"
Jul 27 17:59:27.414057 ip-10-0-133-68 kubenswrapper[2584]: E0727 17:59:27.414039    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7\": container with ID starting with ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7 not found: ID does not exist" containerID="ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7"
Jul 27 17:59:27.414122 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.414068    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7"} err="failed to get container status \"ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7\": rpc error: code = NotFound desc = could not find container \"ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7\": container with ID starting with ca034fda5e620c732c4a085a80f3714860b47e7352cbbf76a68c70f4de1f61d7 not found: ID does not exist"
Jul 27 17:59:27.419817 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.419792    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"]
Jul 27 17:59:27.420892 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.420871    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5mbc\" (UniqueName: \"kubernetes.io/projected/5d6526ad-54e8-448f-bd4f-204aaf53e6db-kube-api-access-l5mbc\") pod \"5d6526ad-54e8-448f-bd4f-204aaf53e6db\" (UID: \"5d6526ad-54e8-448f-bd4f-204aaf53e6db\") "
Jul 27 17:59:27.421065 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.421045    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6526ad-54e8-448f-bd4f-204aaf53e6db-plugins-config-volume\") pod \"5d6526ad-54e8-448f-bd4f-204aaf53e6db\" (UID: \"5d6526ad-54e8-448f-bd4f-204aaf53e6db\") "
Jul 27 17:59:27.421741 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.421413    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:59:27.421741 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.421439    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgdf2\" (UniqueName: \"kubernetes.io/projected/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-kube-api-access-cgdf2\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:59:27.421741 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.421454    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:59:27.421741 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.421555    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d6526ad-54e8-448f-bd4f-204aaf53e6db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5d6526ad-54e8-448f-bd4f-204aaf53e6db" (UID: "5d6526ad-54e8-448f-bd4f-204aaf53e6db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:59:27.423377 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.423355    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d6526ad-54e8-448f-bd4f-204aaf53e6db-kube-api-access-l5mbc" (OuterVolumeSpecName: "kube-api-access-l5mbc") pod "5d6526ad-54e8-448f-bd4f-204aaf53e6db" (UID: "5d6526ad-54e8-448f-bd4f-204aaf53e6db"). InnerVolumeSpecName "kube-api-access-l5mbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:59:27.423842 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.423823    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ab2501-f6f8b9cd-wb42b"]
Jul 27 17:59:27.522942 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.522851    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6526ad-54e8-448f-bd4f-204aaf53e6db-plugins-config-volume\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:59:27.522942 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.522883    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5mbc\" (UniqueName: \"kubernetes.io/projected/5d6526ad-54e8-448f-bd4f-204aaf53e6db-kube-api-access-l5mbc\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 17:59:27.716456 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.716424    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"]
Jul 27 17:59:27.720330 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:27.720304    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ab2501-6f9c6c797b-2bppk"]
Jul 27 17:59:28.823386 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:28.823353    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f" path="/var/lib/kubelet/pods/3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f/volumes"
Jul 27 17:59:28.823801 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:28.823745    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d6526ad-54e8-448f-bd4f-204aaf53e6db" path="/var/lib/kubelet/pods/5d6526ad-54e8-448f-bd4f-204aaf53e6db/volumes"
Jul 27 17:59:31.566136 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:31.566075    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:31.926761 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:31.926659    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:37.868082 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:37.868039    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:37.926667 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:37.926637    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:38.120175 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.120076    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:38.337642 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.337602    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:38.660905 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.660851    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:38.919437 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919346    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"]
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919766    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14a34d56-fc58-4a51-9f68-fbc677ca2e04" containerName="payload-processing"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919779    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="14a34d56-fc58-4a51-9f68-fbc677ca2e04" containerName="payload-processing"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919790    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d6526ad-54e8-448f-bd4f-204aaf53e6db" containerName="payload-pre-processing"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919796    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d6526ad-54e8-448f-bd4f-204aaf53e6db" containerName="payload-pre-processing"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919812    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a529b2eb-7892-4046-bc7c-eca4c26701d6" containerName="maas-api"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919818    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a529b2eb-7892-4046-bc7c-eca4c26701d6" containerName="maas-api"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919824    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f" containerName="maas-api"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919829    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f" containerName="maas-api"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919886    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d416fcb-4ac4-42d1-af5d-5f8e5b9d2e3f" containerName="maas-api"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919893    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="14a34d56-fc58-4a51-9f68-fbc677ca2e04" containerName="payload-processing"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919899    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a529b2eb-7892-4046-bc7c-eca4c26701d6" containerName="maas-api"
Jul 27 17:59:38.919933 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.919905    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d6526ad-54e8-448f-bd4f-204aaf53e6db" containerName="payload-pre-processing"
Jul 27 17:59:38.923192 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.923170    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:38.925976 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.925953    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-617fa4-openshift-default-dockercfg-s69qb\""
Jul 27 17:59:38.936251 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.936225    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"]
Jul 27 17:59:38.963170 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:38.963132    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:39.034992 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.034959    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-token\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.035165 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.035011    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcjzb\" (UniqueName: \"kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-kube-api-access-qcjzb\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.035165 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.035114    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-data\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.035165 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.035143    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-certs\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.035364 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.035171    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-credential-socket\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.035364 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.035276    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-envoy\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.035364 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.035318    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16833af0-41e5-40f0-bee7-bbc018eb5735-istiod-ca-cert\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.035364 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.035353    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-podinfo\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.035619 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.035425    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-socket\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.110842 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.110812    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:39.136180 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136143    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16833af0-41e5-40f0-bee7-bbc018eb5735-istiod-ca-cert\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.136356 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136185    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-podinfo\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.136356 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136225    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-socket\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.136356 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136264    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-token\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.136356 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136292    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjzb\" (UniqueName: \"kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-kube-api-access-qcjzb\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.136356 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136339    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-data\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.136627 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136364    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-certs\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.136627 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136401    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-credential-socket\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.136627 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136442    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-envoy\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.136854 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136822    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-data\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.136919 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.136833    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-credential-socket\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.137143 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.137115    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16833af0-41e5-40f0-bee7-bbc018eb5735-istiod-ca-cert\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.137253 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.137176    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-socket\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.137434 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.137409    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-certs\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.138995 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.138975    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-envoy\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.139210 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.139189    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-podinfo\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.144786 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.144764    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcjzb\" (UniqueName: \"kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-kube-api-access-qcjzb\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.144888 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.144868    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-token\") pod \"e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") " pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.235648 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.235555    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:39.377887 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.377859    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"]
Jul 27 17:59:39.380071 ip-10-0-133-68 kubenswrapper[2584]: W0727 17:59:39.380040    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16833af0_41e5_40f0_bee7_bbc018eb5735.slice/crio-6afc9f43e0b94476d0a73956cf7542f59b7011379cfa7d91e922f25f85f172c8 WatchSource:0}: Error finding container 6afc9f43e0b94476d0a73956cf7542f59b7011379cfa7d91e922f25f85f172c8: Status 404 returned error can't find the container with id 6afc9f43e0b94476d0a73956cf7542f59b7011379cfa7d91e922f25f85f172c8
Jul 27 17:59:39.382260 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.382225    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:59:39.382361 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.382294    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:59:39.382361 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.382323    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 17:59:39.461119 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.461086    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" event={"ID":"16833af0-41e5-40f0-bee7-bbc018eb5735","Type":"ContainerStarted","Data":"ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213"}
Jul 27 17:59:39.461281 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.461145    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" event={"ID":"16833af0-41e5-40f0-bee7-bbc018eb5735","Type":"ContainerStarted","Data":"6afc9f43e0b94476d0a73956cf7542f59b7011379cfa7d91e922f25f85f172c8"}
Jul 27 17:59:39.480882 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:39.480829    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" podStartSLOduration=1.480812003 podStartE2EDuration="1.480812003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:59:39.480371554 +0000 UTC m=+2317.188519896" watchObservedRunningTime="2026-07-27 17:59:39.480812003 +0000 UTC m=+2317.188960321"
Jul 27 17:59:40.236161 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:40.236119    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:40.241346 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:40.241319    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:40.467492 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:40.467462    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:40.468336 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:40.468316    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 17:59:51.352294 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:51.352255    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:51.452437 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:51.452379    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:51.606369 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:51.606269    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:52.030064 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:52.029901    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:52.142035 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:52.142000    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:52.246419 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:52.246382    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:55.813067 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:55.812974    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:56.272846 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:56.272748    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:56.595328 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:56.595295    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:56.746537 ip-10-0-133-68 kubenswrapper[2584]: I0727 17:59:56.746497    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:00.134367 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.134327    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:00.225074 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.225036    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"]
Jul 27 18:00:00.228513 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.228495    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:00.231481 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.231451    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0dcf85-serving-cert\""
Jul 27 18:00:00.232979 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.232954    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:00.242798 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.242766    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"]
Jul 27 18:00:00.262946 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.262904    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63d2f7d3-c1b5-4522-8b45-24d0978121de-openshift-service-ca\") pod \"maas-api-e2e-api-b-0dcf85-54598d4544-6d79t\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:00.263131 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.262968    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d2f7d3-c1b5-4522-8b45-24d0978121de-maas-api-tls\") pod \"maas-api-e2e-api-b-0dcf85-54598d4544-6d79t\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:00.263131 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.263093    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbzc8\" (UniqueName: \"kubernetes.io/projected/63d2f7d3-c1b5-4522-8b45-24d0978121de-kube-api-access-wbzc8\") pod \"maas-api-e2e-api-b-0dcf85-54598d4544-6d79t\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:00.314922 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.314883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:00.363850 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.363813    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d2f7d3-c1b5-4522-8b45-24d0978121de-maas-api-tls\") pod \"maas-api-e2e-api-b-0dcf85-54598d4544-6d79t\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:00.364096 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.363930    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbzc8\" (UniqueName: \"kubernetes.io/projected/63d2f7d3-c1b5-4522-8b45-24d0978121de-kube-api-access-wbzc8\") pod \"maas-api-e2e-api-b-0dcf85-54598d4544-6d79t\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:00.364096 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:00:00.363977    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-serving-cert: secret "maas-api-e2e-api-b-0dcf85-serving-cert" not found
Jul 27 18:00:00.364096 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.363999    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63d2f7d3-c1b5-4522-8b45-24d0978121de-openshift-service-ca\") pod \"maas-api-e2e-api-b-0dcf85-54598d4544-6d79t\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:00.364096 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:00:00.364050    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63d2f7d3-c1b5-4522-8b45-24d0978121de-maas-api-tls podName:63d2f7d3-c1b5-4522-8b45-24d0978121de nodeName:}" failed. No retries permitted until 2026-07-27 18:00:00.864028057 +0000 UTC m=+2338.572176372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63d2f7d3-c1b5-4522-8b45-24d0978121de-maas-api-tls") pod "maas-api-e2e-api-b-0dcf85-54598d4544-6d79t" (UID: "63d2f7d3-c1b5-4522-8b45-24d0978121de") : secret "maas-api-e2e-api-b-0dcf85-serving-cert" not found
Jul 27 18:00:00.364753 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.364730    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63d2f7d3-c1b5-4522-8b45-24d0978121de-openshift-service-ca\") pod \"maas-api-e2e-api-b-0dcf85-54598d4544-6d79t\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:00.379274 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.379225    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbzc8\" (UniqueName: \"kubernetes.io/projected/63d2f7d3-c1b5-4522-8b45-24d0978121de-kube-api-access-wbzc8\") pod \"maas-api-e2e-api-b-0dcf85-54598d4544-6d79t\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:00.692679 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.692641    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:00.868824 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.868781    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d2f7d3-c1b5-4522-8b45-24d0978121de-maas-api-tls\") pod \"maas-api-e2e-api-b-0dcf85-54598d4544-6d79t\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:00.871428 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:00.871406    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d2f7d3-c1b5-4522-8b45-24d0978121de-maas-api-tls\") pod \"maas-api-e2e-api-b-0dcf85-54598d4544-6d79t\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:01.003533 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:01.003436    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:01.099489 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:01.099451    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:01.140979 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:01.140947    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:01.335754 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:01.335717    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"]
Jul 27 18:00:01.337079 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:00:01.337053    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63d2f7d3_c1b5_4522_8b45_24d0978121de.slice/crio-af48b2f7c909feb58725d1fece337985430a600726079f7b938696911cbba4d5 WatchSource:0}: Error finding container af48b2f7c909feb58725d1fece337985430a600726079f7b938696911cbba4d5: Status 404 returned error can't find the container with id af48b2f7c909feb58725d1fece337985430a600726079f7b938696911cbba4d5
Jul 27 18:00:01.338945 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:01.338924    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 18:00:01.589118 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:01.589079    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t" event={"ID":"63d2f7d3-c1b5-4522-8b45-24d0978121de","Type":"ContainerStarted","Data":"af48b2f7c909feb58725d1fece337985430a600726079f7b938696911cbba4d5"}
Jul 27 18:00:02.597570 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:02.597530    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t" event={"ID":"63d2f7d3-c1b5-4522-8b45-24d0978121de","Type":"ContainerStarted","Data":"6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c"}
Jul 27 18:00:02.598054 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:02.597600    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:02.617821 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:02.617752    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t" podStartSLOduration=1.567915722 podStartE2EDuration="2.617729136s" totalImagesPullingTime="1.049813414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:00:02.614142351 +0000 UTC m=+2340.322290683" watchObservedRunningTime="2026-07-27 18:00:02.617729136 +0000 UTC m=+2340.325877455"
Jul 27 18:00:03.604447 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:03.604412    2584 generic.go:356] "Generic (PLEG): container finished" podID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerID="ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213" exitCode=0
Jul 27 18:00:03.604951 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:03.604496    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" event={"ID":"16833af0-41e5-40f0-bee7-bbc018eb5735","Type":"ContainerDied","Data":"ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213"}
Jul 27 18:00:03.605008 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:03.604992    2584 scope.go:117] "RemoveContainer" containerID="ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213"
Jul 27 18:00:03.606011 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:03.605982    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:00:03.606111 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:03.606064    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:00:03.606111 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:03.606115    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:00:04.496392 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:04.496344    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:04.611061 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:04.611024    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" event={"ID":"16833af0-41e5-40f0-bee7-bbc018eb5735","Type":"ContainerStarted","Data":"8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415"}
Jul 27 18:00:05.236454 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:05.236410    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 18:00:05.236730 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:05.236665    2584 patch_prober.go:28] interesting pod/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 27 18:00:05.236855 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:05.236775    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 27 18:00:06.236640 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:06.236601    2584 patch_prober.go:28] interesting pod/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 27 18:00:06.237101 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:06.236662    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 27 18:00:07.236085 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:07.236045    2584 patch_prober.go:28] interesting pod/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 27 18:00:07.236276 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:07.236101    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 27 18:00:08.236406 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:08.236366    2584 patch_prober.go:28] interesting pod/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 27 18:00:08.236857 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:08.236470    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 27 18:00:08.610900 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:08.610871    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:09.240447 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:09.240409    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 18:00:09.240901 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:09.240652    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 18:00:09.241468 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:09.241449    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 18:00:29.050180 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:29.050128    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:29.150099 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:29.150034    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:32.503254 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:32.503221    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:33.217255 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:33.217221    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:36.223044 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:36.223006    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:38.704679 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:38.704619    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:38.845537 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:38.845503    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:38.947132 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:38.947086    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:39.073203 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:39.073165    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"]
Jul 27 18:00:39.073483 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:39.073450    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy" containerID="cri-o://8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415" gracePeriod=30
Jul 27 18:00:39.096649 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:39.096615    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:39.243451 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:39.243412    2584 patch_prober.go:28] interesting pod/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 18:00:39.243631 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:39.243467    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 18:00:39.310302 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:39.310262    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:44.332722 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.332666    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 18:00:44.405410 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405309    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-socket\") pod \"16833af0-41e5-40f0-bee7-bbc018eb5735\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") "
Jul 27 18:00:44.405410 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405380    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-credential-socket\") pod \"16833af0-41e5-40f0-bee7-bbc018eb5735\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") "
Jul 27 18:00:44.405659 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405437    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-data\") pod \"16833af0-41e5-40f0-bee7-bbc018eb5735\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") "
Jul 27 18:00:44.405659 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405462    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-token\") pod \"16833af0-41e5-40f0-bee7-bbc018eb5735\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") "
Jul 27 18:00:44.405659 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405521    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcjzb\" (UniqueName: \"kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-kube-api-access-qcjzb\") pod \"16833af0-41e5-40f0-bee7-bbc018eb5735\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") "
Jul 27 18:00:44.405659 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405537    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-certs\") pod \"16833af0-41e5-40f0-bee7-bbc018eb5735\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") "
Jul 27 18:00:44.405659 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405582    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16833af0-41e5-40f0-bee7-bbc018eb5735-istiod-ca-cert\") pod \"16833af0-41e5-40f0-bee7-bbc018eb5735\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") "
Jul 27 18:00:44.405659 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405630    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-envoy\") pod \"16833af0-41e5-40f0-bee7-bbc018eb5735\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") "
Jul 27 18:00:44.405659 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405640    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "16833af0-41e5-40f0-bee7-bbc018eb5735" (UID: "16833af0-41e5-40f0-bee7-bbc018eb5735"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:00:44.405659 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405658    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-podinfo\") pod \"16833af0-41e5-40f0-bee7-bbc018eb5735\" (UID: \"16833af0-41e5-40f0-bee7-bbc018eb5735\") "
Jul 27 18:00:44.406135 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405762    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "16833af0-41e5-40f0-bee7-bbc018eb5735" (UID: "16833af0-41e5-40f0-bee7-bbc018eb5735"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:00:44.406135 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.405913    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "16833af0-41e5-40f0-bee7-bbc018eb5735" (UID: "16833af0-41e5-40f0-bee7-bbc018eb5735"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:00:44.406135 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.406032    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16833af0-41e5-40f0-bee7-bbc018eb5735-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "16833af0-41e5-40f0-bee7-bbc018eb5735" (UID: "16833af0-41e5-40f0-bee7-bbc018eb5735"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:00:44.406135 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.406073    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-socket\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:44.406135 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.406093    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-credential-socket\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:44.406135 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.406106    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-workload-certs\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:44.406711 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.406670    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-data" (OuterVolumeSpecName: "istio-data") pod "16833af0-41e5-40f0-bee7-bbc018eb5735" (UID: "16833af0-41e5-40f0-bee7-bbc018eb5735"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:00:44.407944 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.407922    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-token" (OuterVolumeSpecName: "istio-token") pod "16833af0-41e5-40f0-bee7-bbc018eb5735" (UID: "16833af0-41e5-40f0-bee7-bbc018eb5735"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:00:44.408169 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.408149    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "16833af0-41e5-40f0-bee7-bbc018eb5735" (UID: "16833af0-41e5-40f0-bee7-bbc018eb5735"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:00:44.408420 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.408389    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-kube-api-access-qcjzb" (OuterVolumeSpecName: "kube-api-access-qcjzb") pod "16833af0-41e5-40f0-bee7-bbc018eb5735" (UID: "16833af0-41e5-40f0-bee7-bbc018eb5735"). InnerVolumeSpecName "kube-api-access-qcjzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:00:44.408516 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.408434    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "16833af0-41e5-40f0-bee7-bbc018eb5735" (UID: "16833af0-41e5-40f0-bee7-bbc018eb5735"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 18:00:44.506904 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.506859    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcjzb\" (UniqueName: \"kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-kube-api-access-qcjzb\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:44.506904 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.506898    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16833af0-41e5-40f0-bee7-bbc018eb5735-istiod-ca-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:44.506904 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.506909    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-envoy\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:44.506904 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.506917    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-podinfo\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:44.507189 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.506925    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-data\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:44.507189 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.506932    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16833af0-41e5-40f0-bee7-bbc018eb5735-istio-token\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:44.825846 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.825810    2584 generic.go:356] "Generic (PLEG): container finished" podID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerID="8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415" exitCode=0
Jul 27 18:00:44.826025 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.825879    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"
Jul 27 18:00:44.826025 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.825900    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" event={"ID":"16833af0-41e5-40f0-bee7-bbc018eb5735","Type":"ContainerDied","Data":"8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415"}
Jul 27 18:00:44.826025 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.825958    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7" event={"ID":"16833af0-41e5-40f0-bee7-bbc018eb5735","Type":"ContainerDied","Data":"6afc9f43e0b94476d0a73956cf7542f59b7011379cfa7d91e922f25f85f172c8"}
Jul 27 18:00:44.826025 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.825980    2584 scope.go:117] "RemoveContainer" containerID="8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415"
Jul 27 18:00:44.836501 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.836288    2584 scope.go:117] "RemoveContainer" containerID="ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213"
Jul 27 18:00:44.844933 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.844911    2584 scope.go:117] "RemoveContainer" containerID="8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415"
Jul 27 18:00:44.845201 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:00:44.845183    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415\": container with ID starting with 8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415 not found: ID does not exist" containerID="8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415"
Jul 27 18:00:44.845303 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.845210    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415"} err="failed to get container status \"8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415\": rpc error: code = NotFound desc = could not find container \"8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415\": container with ID starting with 8c66dc477187f950717d10e20e0f232a8c9a9541a1fbc38204ff92eaa0570415 not found: ID does not exist"
Jul 27 18:00:44.845303 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.845227    2584 scope.go:117] "RemoveContainer" containerID="ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213"
Jul 27 18:00:44.845498 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:00:44.845476    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213\": container with ID starting with ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213 not found: ID does not exist" containerID="ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213"
Jul 27 18:00:44.845545 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.845505    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213"} err="failed to get container status \"ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213\": rpc error: code = NotFound desc = could not find container \"ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213\": container with ID starting with ba24bfe2226a9928a0aab0677c4211939f70f13483340728dbe43932dde1a213 not found: ID does not exist"
Jul 27 18:00:44.851271 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.851234    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"]
Jul 27 18:00:44.854636 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:44.854611    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-617fa4-openshift-default-cdb7b445d-8pnm7"]
Jul 27 18:00:46.823349 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:46.823318    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" path="/var/lib/kubelet/pods/16833af0-41e5-40f0-bee7-bbc018eb5735/volumes"
Jul 27 18:00:54.887756 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:54.887721    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"]
Jul 27 18:00:54.888208 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:54.887963    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t" podUID="63d2f7d3-c1b5-4522-8b45-24d0978121de" containerName="maas-api" containerID="cri-o://6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c" gracePeriod=30
Jul 27 18:00:54.914409 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:54.914379    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:54.985873 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:54.985834    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:55.054728 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.054677    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:55.147460 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.147394    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:55.317779 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.317745    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d2f7d3-c1b5-4522-8b45-24d0978121de-maas-api-tls\") pod \"63d2f7d3-c1b5-4522-8b45-24d0978121de\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") "
Jul 27 18:00:55.317779 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.317798    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbzc8\" (UniqueName: \"kubernetes.io/projected/63d2f7d3-c1b5-4522-8b45-24d0978121de-kube-api-access-wbzc8\") pod \"63d2f7d3-c1b5-4522-8b45-24d0978121de\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") "
Jul 27 18:00:55.318018 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.317835    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63d2f7d3-c1b5-4522-8b45-24d0978121de-openshift-service-ca\") pod \"63d2f7d3-c1b5-4522-8b45-24d0978121de\" (UID: \"63d2f7d3-c1b5-4522-8b45-24d0978121de\") "
Jul 27 18:00:55.318261 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.318236    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63d2f7d3-c1b5-4522-8b45-24d0978121de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63d2f7d3-c1b5-4522-8b45-24d0978121de" (UID: "63d2f7d3-c1b5-4522-8b45-24d0978121de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:00:55.320146 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.320124    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63d2f7d3-c1b5-4522-8b45-24d0978121de-kube-api-access-wbzc8" (OuterVolumeSpecName: "kube-api-access-wbzc8") pod "63d2f7d3-c1b5-4522-8b45-24d0978121de" (UID: "63d2f7d3-c1b5-4522-8b45-24d0978121de"). InnerVolumeSpecName "kube-api-access-wbzc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:00:55.320244 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.320147    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63d2f7d3-c1b5-4522-8b45-24d0978121de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63d2f7d3-c1b5-4522-8b45-24d0978121de" (UID: "63d2f7d3-c1b5-4522-8b45-24d0978121de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:00:55.419416 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.419314    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63d2f7d3-c1b5-4522-8b45-24d0978121de-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:55.419416 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.419349    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbzc8\" (UniqueName: \"kubernetes.io/projected/63d2f7d3-c1b5-4522-8b45-24d0978121de-kube-api-access-wbzc8\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:55.419416 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.419363    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63d2f7d3-c1b5-4522-8b45-24d0978121de-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:00:55.886066 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.886028    2584 generic.go:356] "Generic (PLEG): container finished" podID="63d2f7d3-c1b5-4522-8b45-24d0978121de" containerID="6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c" exitCode=0
Jul 27 18:00:55.886267 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.886077    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"
Jul 27 18:00:55.886267 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.886115    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t" event={"ID":"63d2f7d3-c1b5-4522-8b45-24d0978121de","Type":"ContainerDied","Data":"6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c"}
Jul 27 18:00:55.886267 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.886177    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t" event={"ID":"63d2f7d3-c1b5-4522-8b45-24d0978121de","Type":"ContainerDied","Data":"af48b2f7c909feb58725d1fece337985430a600726079f7b938696911cbba4d5"}
Jul 27 18:00:55.886267 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.886204    2584 scope.go:117] "RemoveContainer" containerID="6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c"
Jul 27 18:00:55.896075 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.895894    2584 scope.go:117] "RemoveContainer" containerID="6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c"
Jul 27 18:00:55.896311 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:00:55.896127    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c\": container with ID starting with 6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c not found: ID does not exist" containerID="6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c"
Jul 27 18:00:55.896311 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.896154    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c"} err="failed to get container status \"6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c\": rpc error: code = NotFound desc = could not find container \"6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c\": container with ID starting with 6be307c91099ec0f8171ff3141c4adeb41cce53c6081612743e267affd10577c not found: ID does not exist"
Jul 27 18:00:55.910103 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.910075    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"]
Jul 27 18:00:55.911975 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:55.911947    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0dcf85-54598d4544-6d79t"]
Jul 27 18:00:56.824320 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:56.824288    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63d2f7d3-c1b5-4522-8b45-24d0978121de" path="/var/lib/kubelet/pods/63d2f7d3-c1b5-4522-8b45-24d0978121de/volumes"
Jul 27 18:00:59.503103 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:59.503065    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:59.863080 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:00:59.863043    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:06.852657 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.852600    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"]
Jul 27 18:01:06.853288 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.853151    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63d2f7d3-c1b5-4522-8b45-24d0978121de" containerName="maas-api"
Jul 27 18:01:06.853288 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.853167    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="63d2f7d3-c1b5-4522-8b45-24d0978121de" containerName="maas-api"
Jul 27 18:01:06.853288 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.853180    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy"
Jul 27 18:01:06.853288 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.853185    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy"
Jul 27 18:01:06.853288 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.853198    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy"
Jul 27 18:01:06.853288 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.853208    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy"
Jul 27 18:01:06.853288 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.853276    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="63d2f7d3-c1b5-4522-8b45-24d0978121de" containerName="maas-api"
Jul 27 18:01:06.853288 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.853293    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy"
Jul 27 18:01:06.853747 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.853301    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="16833af0-41e5-40f0-bee7-bbc018eb5735" containerName="istio-proxy"
Jul 27 18:01:06.856487 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.856467    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:06.859966 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.859941    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-1aa3ff-openshift-default-dockercfg-prgpt\""
Jul 27 18:01:06.869732 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.869703    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"]
Jul 27 18:01:06.929786 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.929711    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-token\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:06.929786 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.929777    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxlvs\" (UniqueName: \"kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-kube-api-access-qxlvs\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:06.930043 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.929813    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-socket\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:06.930043 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.929831    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-envoy\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:06.930043 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.929850    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-data\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:06.930043 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.929883    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-certs\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:06.930043 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.929897    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istiod-ca-cert\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:06.930043 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.929999    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-credential-socket\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:06.930300 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:06.930059    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-podinfo\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.030826 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.030794    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-token\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031036 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.030852    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxlvs\" (UniqueName: \"kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-kube-api-access-qxlvs\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031036 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.030898    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-socket\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031036 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.030921    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-envoy\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031036 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.030943    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-data\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031036 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.030979    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-certs\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031036 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.031005    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istiod-ca-cert\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031300 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.031040    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-credential-socket\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031300 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.031078    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-podinfo\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031415 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.031330    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-socket\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031461 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.031428    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-data\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031627 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.031599    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-credential-socket\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031730 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.031621    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-certs\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.031981 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.031948    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istiod-ca-cert\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.033351 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.033330    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-envoy\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.033606 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.033589    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-podinfo\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.040583 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.040550    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-token\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.042197 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.042171    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxlvs\" (UniqueName: \"kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-kube-api-access-qxlvs\") pod \"e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") " pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.169312 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.169212    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:07.318406 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.318376    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"]
Jul 27 18:01:07.320131 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:01:07.320100    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f8cf84b_8cc0_4d6e_a89d_f9519ffb5c35.slice/crio-d89ff397cabc7d42457ebbecc0d88a36ef2a8064ad3fca311d3e7bf0c7bbf7cb WatchSource:0}: Error finding container d89ff397cabc7d42457ebbecc0d88a36ef2a8064ad3fca311d3e7bf0c7bbf7cb: Status 404 returned error can't find the container with id d89ff397cabc7d42457ebbecc0d88a36ef2a8064ad3fca311d3e7bf0c7bbf7cb
Jul 27 18:01:07.322448 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.322409    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:01:07.322527 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.322508    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:01:07.322569 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.322557    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:01:07.949546 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.949511    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" event={"ID":"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35","Type":"ContainerStarted","Data":"6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7"}
Jul 27 18:01:07.949977 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.949568    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" event={"ID":"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35","Type":"ContainerStarted","Data":"d89ff397cabc7d42457ebbecc0d88a36ef2a8064ad3fca311d3e7bf0c7bbf7cb"}
Jul 27 18:01:07.971516 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:07.971467    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" podStartSLOduration=1.971452513 podStartE2EDuration="1.971452513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:01:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:01:07.970361997 +0000 UTC m=+2405.678510347" watchObservedRunningTime="2026-07-27 18:01:07.971452513 +0000 UTC m=+2405.679600830"
Jul 27 18:01:08.170243 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:08.170203    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:08.175375 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:08.175347    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:08.851911 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:08.851867    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:08.951197 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:08.951159    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:08.954551 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:08.954529    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:08.955393 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:08.955377    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:09.989066 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:09.989018    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:10.100659 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.100619    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:10.177940 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.177902    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:10.422423 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.422382    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:10.676552 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.676451    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:10.774366 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.774321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:10.802177 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.802135    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"]
Jul 27 18:01:10.805880 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.805846    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:01:10.809088 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.809061    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-1aa3ff\""
Jul 27 18:01:10.809231 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.809143    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-1aa3ff-dockercfg-6sn4g\""
Jul 27 18:01:10.815212 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.815187    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"]
Jul 27 18:01:10.867467 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.867431    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8zpb\" (UniqueName: \"kubernetes.io/projected/c0e4b412-5c96-42a4-948d-1b53f9622f98-kube-api-access-k8zpb\") pod \"payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25\" (UID: \"c0e4b412-5c96-42a4-948d-1b53f9622f98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:01:10.867678 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.867504    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e4b412-5c96-42a4-948d-1b53f9622f98-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25\" (UID: \"c0e4b412-5c96-42a4-948d-1b53f9622f98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:01:10.968273 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.968177    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e4b412-5c96-42a4-948d-1b53f9622f98-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25\" (UID: \"c0e4b412-5c96-42a4-948d-1b53f9622f98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:01:10.968453 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.968298    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8zpb\" (UniqueName: \"kubernetes.io/projected/c0e4b412-5c96-42a4-948d-1b53f9622f98-kube-api-access-k8zpb\") pod \"payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25\" (UID: \"c0e4b412-5c96-42a4-948d-1b53f9622f98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:01:10.969005 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.968982    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e4b412-5c96-42a4-948d-1b53f9622f98-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25\" (UID: \"c0e4b412-5c96-42a4-948d-1b53f9622f98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:01:10.977504 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:10.977472    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8zpb\" (UniqueName: \"kubernetes.io/projected/c0e4b412-5c96-42a4-948d-1b53f9622f98-kube-api-access-k8zpb\") pod \"payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25\" (UID: \"c0e4b412-5c96-42a4-948d-1b53f9622f98\") " pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:01:11.118504 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:11.118462    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:01:11.260315 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:11.260208    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"]
Jul 27 18:01:11.262936 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:01:11.262908    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e4b412_5c96_42a4_948d_1b53f9622f98.slice/crio-cf4780d5f001664f2ed94cf6c2ca8e3236839a32c12dba8c6e0f68e8c2f4e68f WatchSource:0}: Error finding container cf4780d5f001664f2ed94cf6c2ca8e3236839a32c12dba8c6e0f68e8c2f4e68f: Status 404 returned error can't find the container with id cf4780d5f001664f2ed94cf6c2ca8e3236839a32c12dba8c6e0f68e8c2f4e68f
Jul 27 18:01:11.971357 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:11.971322    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25" event={"ID":"c0e4b412-5c96-42a4-948d-1b53f9622f98","Type":"ContainerStarted","Data":"06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3"}
Jul 27 18:01:11.971566 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:11.971378    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25" event={"ID":"c0e4b412-5c96-42a4-948d-1b53f9622f98","Type":"ContainerStarted","Data":"cf4780d5f001664f2ed94cf6c2ca8e3236839a32c12dba8c6e0f68e8c2f4e68f"}
Jul 27 18:01:11.971566 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:11.971408    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:01:11.993998 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:11.993945    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25" podStartSLOduration=1.9939257929999998 podStartE2EDuration="1.993925793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:01:11.993354354 +0000 UTC m=+2409.701502688" watchObservedRunningTime="2026-07-27 18:01:11.993925793 +0000 UTC m=+2409.702074111"
Jul 27 18:01:14.385061 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:14.385024    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:14.727483 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:14.727380    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:15.146117 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:15.146085    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:15.336997 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:15.336962    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:18.637655 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.637607    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:18.717372 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.715672    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"]
Jul 27 18:01:18.721429 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.721402    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:18.724335 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.724306    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"]
Jul 27 18:01:18.724588 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.724413    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ed5396-serving-cert\""
Jul 27 18:01:18.743240 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.743197    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:18.839023 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.838984    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:18.844182 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.844159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtdxv\" (UniqueName: \"kubernetes.io/projected/a2e369c5-e909-472c-ad18-c6b9646773d9-kube-api-access-xtdxv\") pod \"maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:18.844286 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.844212    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e369c5-e909-472c-ad18-c6b9646773d9-maas-api-tls\") pod \"maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:18.844286 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.844239    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2e369c5-e909-472c-ad18-c6b9646773d9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:18.945222 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.945131    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtdxv\" (UniqueName: \"kubernetes.io/projected/a2e369c5-e909-472c-ad18-c6b9646773d9-kube-api-access-xtdxv\") pod \"maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:18.945222 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.945197    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e369c5-e909-472c-ad18-c6b9646773d9-maas-api-tls\") pod \"maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:18.945420 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.945229    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2e369c5-e909-472c-ad18-c6b9646773d9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:18.945420 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:01:18.945338    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-serving-cert: secret "maas-api-e2e-shared-b-ed5396-serving-cert" not found
Jul 27 18:01:18.945502 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:01:18.945421    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2e369c5-e909-472c-ad18-c6b9646773d9-maas-api-tls podName:a2e369c5-e909-472c-ad18-c6b9646773d9 nodeName:}" failed. No retries permitted until 2026-07-27 18:01:19.445400242 +0000 UTC m=+2417.153548538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a2e369c5-e909-472c-ad18-c6b9646773d9-maas-api-tls") pod "maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs" (UID: "a2e369c5-e909-472c-ad18-c6b9646773d9") : secret "maas-api-e2e-shared-b-ed5396-serving-cert" not found
Jul 27 18:01:18.945947 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.945926    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2e369c5-e909-472c-ad18-c6b9646773d9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:18.957780 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:18.957747    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtdxv\" (UniqueName: \"kubernetes.io/projected/a2e369c5-e909-472c-ad18-c6b9646773d9-kube-api-access-xtdxv\") pod \"maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:19.050881 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:19.050840    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:19.405112 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:19.405074    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:19.451580 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:19.451543    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e369c5-e909-472c-ad18-c6b9646773d9-maas-api-tls\") pod \"maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:19.454206 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:19.454180    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e369c5-e909-472c-ad18-c6b9646773d9-maas-api-tls\") pod \"maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:19.505978 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:19.505938    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:19.632989 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:19.632952    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:19.807978 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:19.807946    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"]
Jul 27 18:01:19.809951 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:01:19.809911    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e369c5_e909_472c_ad18_c6b9646773d9.slice/crio-5e4a53ee9d25c6a4b2abeda0be20ef4a532dfae7aeac78fa88dbd00b3dc23dab WatchSource:0}: Error finding container 5e4a53ee9d25c6a4b2abeda0be20ef4a532dfae7aeac78fa88dbd00b3dc23dab: Status 404 returned error can't find the container with id 5e4a53ee9d25c6a4b2abeda0be20ef4a532dfae7aeac78fa88dbd00b3dc23dab
Jul 27 18:01:20.016785 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:20.016677    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs" event={"ID":"a2e369c5-e909-472c-ad18-c6b9646773d9","Type":"ContainerStarted","Data":"5e4a53ee9d25c6a4b2abeda0be20ef4a532dfae7aeac78fa88dbd00b3dc23dab"}
Jul 27 18:01:22.978554 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:22.978521    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:01:23.033498 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:23.033454    2584 generic.go:356] "Generic (PLEG): container finished" podID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerID="6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7" exitCode=0
Jul 27 18:01:23.033675 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:23.033528    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" event={"ID":"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35","Type":"ContainerDied","Data":"6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7"}
Jul 27 18:01:23.034134 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:23.034109    2584 scope.go:117] "RemoveContainer" containerID="6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7"
Jul 27 18:01:23.035381 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:23.035343    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:01:23.035462 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:23.035447    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:01:23.035505 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:23.035484    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:01:23.042074 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:23.042049    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:01:23.955015 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:23.954976    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:24.041224 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:24.041183    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" event={"ID":"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35","Type":"ContainerStarted","Data":"b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8"}
Jul 27 18:01:24.042977 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:24.042948    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs" event={"ID":"a2e369c5-e909-472c-ad18-c6b9646773d9","Type":"ContainerStarted","Data":"f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9"}
Jul 27 18:01:24.043102 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:24.043056    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:24.092840 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:24.092765    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs" podStartSLOduration=2.565289054 podStartE2EDuration="6.092740441s" totalImagesPullingTime="3.527451387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:01:24.090372542 +0000 UTC m=+2421.798520861" watchObservedRunningTime="2026-07-27 18:01:24.092740441 +0000 UTC m=+2421.800888760"
Jul 27 18:01:24.170676 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:24.170635    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:24.170676 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:24.170655    2584 patch_prober.go:28] interesting pod/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 27 18:01:24.170967 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:24.170735    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" podUID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 27 18:01:25.169638 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:25.169589    2584 patch_prober.go:28] interesting pod/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 27 18:01:25.170211 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:25.169659    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" podUID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 27 18:01:26.170494 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:26.170454    2584 patch_prober.go:28] interesting pod/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 27 18:01:26.171153 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:26.171119    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" podUID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 27 18:01:27.170227 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:27.170178    2584 patch_prober.go:28] interesting pod/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 27 18:01:27.170451 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:27.170260    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" podUID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 27 18:01:28.173520 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:28.173486    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:28.174025 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:28.173906    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:28.174460 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:28.174439    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:01:30.054450 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:30.054420    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:01:42.030414 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:01:42.030381    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:02:16.271266 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:02:16.271211    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:02:56.803645 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:02:56.803528    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:03:03.211908 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:03:03.211849    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:04:04.390605 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:04:04.390548    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:04:04.492466 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:04:04.492430    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:06:15.798295 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:06:15.798198    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:06:15.888611 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:06:15.888578    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:06:51.062097 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:06:51.062058    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:06:51.141318 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:06:51.141267    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:06:54.124572 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:06:54.124535    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:06:54.225192 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:06:54.225156    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:06:54.366790 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:06:54.366753    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:06:55.288589 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:06:55.288554    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:07:04.807634 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:07:04.807606    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:07:28.011499 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:07:28.011418    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:08:29.971791 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:08:29.971740    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:08:32.154526 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:08:32.154491    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:08:50.584306 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:08:50.584251    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:09:40.172816 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:09:40.172778    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:10:49.695426 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:10:49.695348    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:10:50.585013 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:10:50.584971    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:09.614464 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:09.614419    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:09.927183 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:09.927088    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:10.463441 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:10.463409    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:12.792053 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:12.792017    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:12.910608 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:12.910574    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:12.994376 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:12.994339    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:13.559239 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.559203    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:13.676985 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.676951    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:13.800863 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.800828    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:13.837446 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.837342    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"]
Jul 27 18:11:13.841122 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.841099    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:11:13.844643 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.844622    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-7985e5-dockercfg-262bt\""
Jul 27 18:11:13.844772 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.844657    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-7985e5\""
Jul 27 18:11:13.858730 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.858680    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"]
Jul 27 18:11:13.912660 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.912619    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg9lw\" (UniqueName: \"kubernetes.io/projected/c5bdb232-185b-4963-b0a2-099c4549af19-kube-api-access-sg9lw\") pod \"payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj\" (UID: \"c5bdb232-185b-4963-b0a2-099c4549af19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:11:13.912885 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.912741    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bdb232-185b-4963-b0a2-099c4549af19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj\" (UID: \"c5bdb232-185b-4963-b0a2-099c4549af19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:11:13.921302 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.921269    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"]
Jul 27 18:11:13.925169 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.925141    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:11:13.939563 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:13.939528    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"]
Jul 27 18:11:14.013538 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.013504    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbb1321c-2dff-4612-b2b0-d829ab6db438-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd\" (UID: \"bbb1321c-2dff-4612-b2b0-d829ab6db438\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:11:14.013538 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.013544    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bdb232-185b-4963-b0a2-099c4549af19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj\" (UID: \"c5bdb232-185b-4963-b0a2-099c4549af19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:11:14.013854 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.013638    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmqdr\" (UniqueName: \"kubernetes.io/projected/bbb1321c-2dff-4612-b2b0-d829ab6db438-kube-api-access-jmqdr\") pod \"payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd\" (UID: \"bbb1321c-2dff-4612-b2b0-d829ab6db438\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:11:14.013854 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.013720    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg9lw\" (UniqueName: \"kubernetes.io/projected/c5bdb232-185b-4963-b0a2-099c4549af19-kube-api-access-sg9lw\") pod \"payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj\" (UID: \"c5bdb232-185b-4963-b0a2-099c4549af19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:11:14.014218 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.014201    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bdb232-185b-4963-b0a2-099c4549af19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj\" (UID: \"c5bdb232-185b-4963-b0a2-099c4549af19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:11:14.023917 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.023891    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg9lw\" (UniqueName: \"kubernetes.io/projected/c5bdb232-185b-4963-b0a2-099c4549af19-kube-api-access-sg9lw\") pod \"payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj\" (UID: \"c5bdb232-185b-4963-b0a2-099c4549af19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:11:14.114788 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.114666    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbb1321c-2dff-4612-b2b0-d829ab6db438-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd\" (UID: \"bbb1321c-2dff-4612-b2b0-d829ab6db438\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:11:14.114978 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.114795    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmqdr\" (UniqueName: \"kubernetes.io/projected/bbb1321c-2dff-4612-b2b0-d829ab6db438-kube-api-access-jmqdr\") pod \"payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd\" (UID: \"bbb1321c-2dff-4612-b2b0-d829ab6db438\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:11:14.115437 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.115418    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbb1321c-2dff-4612-b2b0-d829ab6db438-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd\" (UID: \"bbb1321c-2dff-4612-b2b0-d829ab6db438\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:11:14.126476 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.126450    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmqdr\" (UniqueName: \"kubernetes.io/projected/bbb1321c-2dff-4612-b2b0-d829ab6db438-kube-api-access-jmqdr\") pod \"payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd\" (UID: \"bbb1321c-2dff-4612-b2b0-d829ab6db438\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:11:14.152042 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.152003    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:11:14.241273 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.241219    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:11:14.297009 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.296979    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"]
Jul 27 18:11:14.300631 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:11:14.300585    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5bdb232_185b_4963_b0a2_099c4549af19.slice/crio-2443962c8d72d1115f045d0e0c3103adf34301400f7f369d54d50c81b296c511 WatchSource:0}: Error finding container 2443962c8d72d1115f045d0e0c3103adf34301400f7f369d54d50c81b296c511: Status 404 returned error can't find the container with id 2443962c8d72d1115f045d0e0c3103adf34301400f7f369d54d50c81b296c511
Jul 27 18:11:14.403434 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:14.403408    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"]
Jul 27 18:11:14.405747 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:11:14.405718    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb1321c_2dff_4612_b2b0_d829ab6db438.slice/crio-5adfb9eb100ac731158f7e599c5c61ae3ef1068b172c98507f7768f608ef9064 WatchSource:0}: Error finding container 5adfb9eb100ac731158f7e599c5c61ae3ef1068b172c98507f7768f608ef9064: Status 404 returned error can't find the container with id 5adfb9eb100ac731158f7e599c5c61ae3ef1068b172c98507f7768f608ef9064
Jul 27 18:11:15.116236 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:15.116185    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj" event={"ID":"c5bdb232-185b-4963-b0a2-099c4549af19","Type":"ContainerStarted","Data":"b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35"}
Jul 27 18:11:15.116723 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:15.116352    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:11:15.116723 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:15.116382    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj" event={"ID":"c5bdb232-185b-4963-b0a2-099c4549af19","Type":"ContainerStarted","Data":"2443962c8d72d1115f045d0e0c3103adf34301400f7f369d54d50c81b296c511"}
Jul 27 18:11:15.117813 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:15.117791    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd" event={"ID":"bbb1321c-2dff-4612-b2b0-d829ab6db438","Type":"ContainerStarted","Data":"fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3"}
Jul 27 18:11:15.117911 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:15.117832    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd" event={"ID":"bbb1321c-2dff-4612-b2b0-d829ab6db438","Type":"ContainerStarted","Data":"5adfb9eb100ac731158f7e599c5c61ae3ef1068b172c98507f7768f608ef9064"}
Jul 27 18:11:15.117911 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:15.117857    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:11:15.134985 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:15.134923    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj" podStartSLOduration=2.134908232 podStartE2EDuration="2.134908232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:11:15.133339086 +0000 UTC m=+3012.841487416" watchObservedRunningTime="2026-07-27 18:11:15.134908232 +0000 UTC m=+3012.843056551"
Jul 27 18:11:15.149514 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:15.149448    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd" podStartSLOduration=2.149428573 podStartE2EDuration="2.149428573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:11:15.149185534 +0000 UTC m=+3012.857333853" watchObservedRunningTime="2026-07-27 18:11:15.149428573 +0000 UTC m=+3012.857576892"
Jul 27 18:11:17.265865 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:17.265828    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:26.125772 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:26.125734    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:11:26.126401 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:26.125819    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:11:28.280832 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:28.280786    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:28.595927 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:28.595884    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:29.039866 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:29.039781    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:32.059936 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.059902    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:32.159417 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.159377    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:32.176475 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.176436    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"]
Jul 27 18:11:32.181434 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.181406    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.184485 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.184455    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-fbef1a-serving-cert\""
Jul 27 18:11:32.194081 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.194053    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"]
Jul 27 18:11:32.254432 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.254393    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:32.314600 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.314499    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbt9g\" (UniqueName: \"kubernetes.io/projected/963f1f41-6d40-4c97-a3dd-03745d101b6e-kube-api-access-vbt9g\") pod \"maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.314600 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.314574    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963f1f41-6d40-4c97-a3dd-03745d101b6e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.314858 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.314618    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963f1f41-6d40-4c97-a3dd-03745d101b6e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.416277 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.416239    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbt9g\" (UniqueName: \"kubernetes.io/projected/963f1f41-6d40-4c97-a3dd-03745d101b6e-kube-api-access-vbt9g\") pod \"maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.416472 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.416297    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963f1f41-6d40-4c97-a3dd-03745d101b6e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.416472 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.416351    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963f1f41-6d40-4c97-a3dd-03745d101b6e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.416548 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:11:32.416464    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-serving-cert: secret "maas-api-e2e-ipp-b-fbef1a-serving-cert" not found
Jul 27 18:11:32.416548 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:11:32.416530    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963f1f41-6d40-4c97-a3dd-03745d101b6e-maas-api-tls podName:963f1f41-6d40-4c97-a3dd-03745d101b6e nodeName:}" failed. No retries permitted until 2026-07-27 18:11:32.916512979 +0000 UTC m=+3030.624661282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/963f1f41-6d40-4c97-a3dd-03745d101b6e-maas-api-tls") pod "maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5" (UID: "963f1f41-6d40-4c97-a3dd-03745d101b6e") : secret "maas-api-e2e-ipp-b-fbef1a-serving-cert" not found
Jul 27 18:11:32.417038 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.417020    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963f1f41-6d40-4c97-a3dd-03745d101b6e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.426592 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.426570    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbt9g\" (UniqueName: \"kubernetes.io/projected/963f1f41-6d40-4c97-a3dd-03745d101b6e-kube-api-access-vbt9g\") pod \"maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.718516 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.718431    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:32.879004 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.878969    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:32.923010 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.922976    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963f1f41-6d40-4c97-a3dd-03745d101b6e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.925779 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.925748    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963f1f41-6d40-4c97-a3dd-03745d101b6e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:32.979214 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:32.979123    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:33.023523 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.023484    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"]
Jul 27 18:11:33.027514 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.027493    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:11:33.030630 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.030603    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-fbef1a\""
Jul 27 18:11:33.030630 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.030603    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-fbef1a-dockercfg-994hj\""
Jul 27 18:11:33.035182 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.035152    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"]
Jul 27 18:11:33.094025 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.093984    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:33.125134 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.125091    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkkzn\" (UniqueName: \"kubernetes.io/projected/5f39721d-3a00-4c50-a2db-c666f8e51003-kube-api-access-tkkzn\") pod \"payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx\" (UID: \"5f39721d-3a00-4c50-a2db-c666f8e51003\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:11:33.125320 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.125172    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f39721d-3a00-4c50-a2db-c666f8e51003-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx\" (UID: \"5f39721d-3a00-4c50-a2db-c666f8e51003\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:11:33.226964 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.226931    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkkzn\" (UniqueName: \"kubernetes.io/projected/5f39721d-3a00-4c50-a2db-c666f8e51003-kube-api-access-tkkzn\") pod \"payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx\" (UID: \"5f39721d-3a00-4c50-a2db-c666f8e51003\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:11:33.227160 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.227016    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f39721d-3a00-4c50-a2db-c666f8e51003-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx\" (UID: \"5f39721d-3a00-4c50-a2db-c666f8e51003\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:11:33.228068 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.228037    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f39721d-3a00-4c50-a2db-c666f8e51003-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx\" (UID: \"5f39721d-3a00-4c50-a2db-c666f8e51003\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:11:33.236629 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.236546    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkkzn\" (UniqueName: \"kubernetes.io/projected/5f39721d-3a00-4c50-a2db-c666f8e51003-kube-api-access-tkkzn\") pod \"payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx\" (UID: \"5f39721d-3a00-4c50-a2db-c666f8e51003\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:11:33.255775 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.255746    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"]
Jul 27 18:11:33.257457 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:11:33.257428    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod963f1f41_6d40_4c97_a3dd_03745d101b6e.slice/crio-2078e90b58f503575ad9a00eef499f507a044bfcc59365a759e562bbf8bd4c1c WatchSource:0}: Error finding container 2078e90b58f503575ad9a00eef499f507a044bfcc59365a759e562bbf8bd4c1c: Status 404 returned error can't find the container with id 2078e90b58f503575ad9a00eef499f507a044bfcc59365a759e562bbf8bd4c1c
Jul 27 18:11:33.258871 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.258851    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 18:11:33.341076 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.341036    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:11:33.486054 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:33.486025    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"]
Jul 27 18:11:33.487938 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:11:33.487860    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f39721d_3a00_4c50_a2db_c666f8e51003.slice/crio-0a97857bdbdf5e9ae859f1666602401daaa389261100a8ed5e499745eb6b8b30 WatchSource:0}: Error finding container 0a97857bdbdf5e9ae859f1666602401daaa389261100a8ed5e499745eb6b8b30: Status 404 returned error can't find the container with id 0a97857bdbdf5e9ae859f1666602401daaa389261100a8ed5e499745eb6b8b30
Jul 27 18:11:34.223058 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:34.222949    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5" event={"ID":"963f1f41-6d40-4c97-a3dd-03745d101b6e","Type":"ContainerStarted","Data":"e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab"}
Jul 27 18:11:34.223058 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:34.223013    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5" event={"ID":"963f1f41-6d40-4c97-a3dd-03745d101b6e","Type":"ContainerStarted","Data":"2078e90b58f503575ad9a00eef499f507a044bfcc59365a759e562bbf8bd4c1c"}
Jul 27 18:11:34.223058 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:34.223045    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:34.224541 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:34.224512    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx" event={"ID":"5f39721d-3a00-4c50-a2db-c666f8e51003","Type":"ContainerStarted","Data":"21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028"}
Jul 27 18:11:34.224711 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:34.224552    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx" event={"ID":"5f39721d-3a00-4c50-a2db-c666f8e51003","Type":"ContainerStarted","Data":"0a97857bdbdf5e9ae859f1666602401daaa389261100a8ed5e499745eb6b8b30"}
Jul 27 18:11:34.224711 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:34.224646    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:11:34.242900 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:34.242706    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5" podStartSLOduration=1.63247713 podStartE2EDuration="2.242669026s" totalImagesPullingTime="610.191896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:11:34.242442957 +0000 UTC m=+3031.950591279" watchObservedRunningTime="2026-07-27 18:11:34.242669026 +0000 UTC m=+3031.950817345"
Jul 27 18:11:34.266041 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:34.265976    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx" podStartSLOduration=1.265955416 podStartE2EDuration="1.265955416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:11:34.263400528 +0000 UTC m=+3031.971548848" watchObservedRunningTime="2026-07-27 18:11:34.265955416 +0000 UTC m=+3031.974103741"
Jul 27 18:11:36.675963 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:36.675917    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:39.264272 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:39.264236    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:39.368001 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:39.367964    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:40.235959 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:40.235932    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:11:45.232935 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:45.232897    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:11:47.745348 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:47.745300    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:54.123594 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:11:54.123512    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx_5f39721d-3a00-4c50-a2db-c666f8e51003/payload-processing/0.log"
Jul 27 18:12:17.682696 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:17.682663    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:17.783146 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:17.783108    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:19.981895 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:19.981864    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:20.081772 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:20.081738    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:20.209963 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:20.209927    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:41.662993 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.662957    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:41.677153 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.677111    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c64ccc968-j5t26"]
Jul 27 18:12:41.682938 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.682909    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.702760 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.702732    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c64ccc968-j5t26"]
Jul 27 18:12:41.760099 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.760062    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-oauth-config\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.760099 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.760100    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-serving-cert\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.760318 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.760183    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-oauth-serving-cert\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.760318 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.760237    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-config\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.760318 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.760290    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-trusted-ca-bundle\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.760421 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.760324    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4k5v\" (UniqueName: \"kubernetes.io/projected/87c5bc5d-4716-4208-9c07-e558a94dd7ad-kube-api-access-f4k5v\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.760421 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.760386    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-service-ca\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.861462 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.861417    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-service-ca\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.861650 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.861486    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-oauth-config\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.861650 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.861514    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-serving-cert\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.861650 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.861600    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-oauth-serving-cert\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.861650 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.861640    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-config\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.861906 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.861741    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-trusted-ca-bundle\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.861906 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.861792    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4k5v\" (UniqueName: \"kubernetes.io/projected/87c5bc5d-4716-4208-9c07-e558a94dd7ad-kube-api-access-f4k5v\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.862310 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.862281    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-service-ca\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.862863 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.862839    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-oauth-serving-cert\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.863199 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.863149    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-trusted-ca-bundle\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.863487 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.863361    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-config\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.865256 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.865234    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-oauth-config\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.865480 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.865445    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-serving-cert\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.886640 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.886605    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4k5v\" (UniqueName: \"kubernetes.io/projected/87c5bc5d-4716-4208-9c07-e558a94dd7ad-kube-api-access-f4k5v\") pod \"console-6c64ccc968-j5t26\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") " pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:41.978474 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.978375    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:41.995477 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:41.995442    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:42.087235 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:42.086972    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:42.186449 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:42.186420    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:42.220611 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:42.220564    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c64ccc968-j5t26"]
Jul 27 18:12:42.230712 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:12:42.230601    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c5bc5d_4716_4208_9c07_e558a94dd7ad.slice/crio-b28f61cd43c4f9f2661230e67e903c41d13755f7a91b5b78e492ed0a3f04c4ac WatchSource:0}: Error finding container b28f61cd43c4f9f2661230e67e903c41d13755f7a91b5b78e492ed0a3f04c4ac: Status 404 returned error can't find the container with id b28f61cd43c4f9f2661230e67e903c41d13755f7a91b5b78e492ed0a3f04c4ac
Jul 27 18:12:42.367528 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:42.367491    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:42.567035 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:42.566996    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:42.586902 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:42.586859    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c64ccc968-j5t26" event={"ID":"87c5bc5d-4716-4208-9c07-e558a94dd7ad","Type":"ContainerStarted","Data":"5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91"}
Jul 27 18:12:42.587090 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:42.586917    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c64ccc968-j5t26" event={"ID":"87c5bc5d-4716-4208-9c07-e558a94dd7ad","Type":"ContainerStarted","Data":"b28f61cd43c4f9f2661230e67e903c41d13755f7a91b5b78e492ed0a3f04c4ac"}
Jul 27 18:12:42.608748 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:42.608697    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c64ccc968-j5t26" podStartSLOduration=1.608664917 podStartE2EDuration="1.608664917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:12:42.606822342 +0000 UTC m=+3100.314970661" watchObservedRunningTime="2026-07-27 18:12:42.608664917 +0000 UTC m=+3100.316813303"
Jul 27 18:12:42.899073 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:42.898973    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:42.994187 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:42.994150    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:43.295021 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:43.294920    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:43.395101 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:43.395052    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:43.899367 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:43.899328    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:43.995008 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:43.994968    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:46.550731 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:46.550677    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:46.650962 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:46.650926    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:51.996525 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:51.996463    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:51.996942 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:51.996591    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:52.001833 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:52.001807    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:52.647185 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:52.647153    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:12:52.699871 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:52.699834    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b95f967bb-ptmvb"]
Jul 27 18:12:52.816115 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:52.816078    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:52.916462 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:52.916368    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:53.016793 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:53.016757    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:53.460914 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:53.460878    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:53.615249 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:53.615208    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:53.979109 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:53.979076    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:54.137791 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.137755    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:54.275245 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.275159    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:54.407831 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.407787    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:54.552333 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.552246    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:54.664026 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.663990    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67488b6847-sjmwx"]
Jul 27 18:12:54.668937 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.668916    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.681786 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.681762    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67488b6847-sjmwx"]
Jul 27 18:12:54.798258 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.798211    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-console-config\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.798509 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.798301    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a7802a1-29ba-40da-9764-1308555af6d5-console-oauth-config\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.798509 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.798327    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a7802a1-29ba-40da-9764-1308555af6d5-console-serving-cert\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.798509 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.798393    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-trusted-ca-bundle\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.798509 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.798478    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-service-ca\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.798812 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.798532    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-oauth-serving-cert\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.798812 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.798614    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw7zc\" (UniqueName: \"kubernetes.io/projected/8a7802a1-29ba-40da-9764-1308555af6d5-kube-api-access-pw7zc\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.798812 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.798633    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:54.899282 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.899249    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-service-ca\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.899490 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.899329    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-oauth-serving-cert\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.899490 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.899377    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw7zc\" (UniqueName: \"kubernetes.io/projected/8a7802a1-29ba-40da-9764-1308555af6d5-kube-api-access-pw7zc\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.899609 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.899501    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-console-config\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.899609 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.899542    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a7802a1-29ba-40da-9764-1308555af6d5-console-oauth-config\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.899609 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.899575    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a7802a1-29ba-40da-9764-1308555af6d5-console-serving-cert\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.899795 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.899627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-trusted-ca-bundle\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.900215 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.900183    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-service-ca\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.900215 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.900201    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-oauth-serving-cert\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.900424 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.900253    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-console-config\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.900526 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.900506    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a7802a1-29ba-40da-9764-1308555af6d5-trusted-ca-bundle\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.902308 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.902286    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a7802a1-29ba-40da-9764-1308555af6d5-console-serving-cert\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.902418 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.902311    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a7802a1-29ba-40da-9764-1308555af6d5-console-oauth-config\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.911285 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.911262    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw7zc\" (UniqueName: \"kubernetes.io/projected/8a7802a1-29ba-40da-9764-1308555af6d5-kube-api-access-pw7zc\") pod \"console-67488b6847-sjmwx\" (UID: \"8a7802a1-29ba-40da-9764-1308555af6d5\") " pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:54.921887 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.921860    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:54.979646 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:54.979608    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:12:55.048428 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:55.048389    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:55.127492 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:55.127405    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67488b6847-sjmwx"]
Jul 27 18:12:55.130143 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:12:55.130110    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7802a1_29ba_40da_9764_1308555af6d5.slice/crio-3055085facf39f3348ad9b8f0c59d4a27bc75f3ad757a104fbd4c490358f5669 WatchSource:0}: Error finding container 3055085facf39f3348ad9b8f0c59d4a27bc75f3ad757a104fbd4c490358f5669: Status 404 returned error can't find the container with id 3055085facf39f3348ad9b8f0c59d4a27bc75f3ad757a104fbd4c490358f5669
Jul 27 18:12:55.663044 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:55.663009    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67488b6847-sjmwx" event={"ID":"8a7802a1-29ba-40da-9764-1308555af6d5","Type":"ContainerStarted","Data":"54f318fe8e8a7798de40cb6e320f9b522c2c2af77041d5c18c40a2f0b71b1711"}
Jul 27 18:12:55.663476 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:55.663068    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67488b6847-sjmwx" event={"ID":"8a7802a1-29ba-40da-9764-1308555af6d5","Type":"ContainerStarted","Data":"3055085facf39f3348ad9b8f0c59d4a27bc75f3ad757a104fbd4c490358f5669"}
Jul 27 18:12:55.694730 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:55.694659    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67488b6847-sjmwx" podStartSLOduration=1.69463998 podStartE2EDuration="1.69463998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:12:55.692999793 +0000 UTC m=+3113.401148111" watchObservedRunningTime="2026-07-27 18:12:55.69463998 +0000 UTC m=+3113.402788298"
Jul 27 18:12:55.808169 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:55.808134    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:56.198567 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:56.198531    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:56.293349 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:12:56.293318    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:02.056435 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:02.056403    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:02.157196 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:02.157161    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:03.192016 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:03.191975    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:03.342902 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:03.342834    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:04.980737 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:04.980674    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:13:04.981140 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:04.980768    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:13:04.985614 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:04.985584    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:13:05.727902 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:05.727873    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67488b6847-sjmwx"
Jul 27 18:13:05.777983 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:05.777946    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c64ccc968-j5t26"]
Jul 27 18:13:05.935573 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:05.935537    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:06.080568 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:06.080538    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:06.158178 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:06.158144    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:17.725746 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:17.725672    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b95f967bb-ptmvb" podUID="af864fee-581d-43c8-ae03-6a9a04e8a577" containerName="console" containerID="cri-o://b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51" gracePeriod=15
Jul 27 18:13:17.994707 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:17.994668    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 18:13:18.042001 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.041966    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-oauth-config\") pod \"af864fee-581d-43c8-ae03-6a9a04e8a577\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") "
Jul 27 18:13:18.042196 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.042025    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76qcl\" (UniqueName: \"kubernetes.io/projected/af864fee-581d-43c8-ae03-6a9a04e8a577-kube-api-access-76qcl\") pod \"af864fee-581d-43c8-ae03-6a9a04e8a577\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") "
Jul 27 18:13:18.042196 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.042046    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-service-ca\") pod \"af864fee-581d-43c8-ae03-6a9a04e8a577\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") "
Jul 27 18:13:18.042196 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.042067    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-console-config\") pod \"af864fee-581d-43c8-ae03-6a9a04e8a577\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") "
Jul 27 18:13:18.042196 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.042090    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-oauth-serving-cert\") pod \"af864fee-581d-43c8-ae03-6a9a04e8a577\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") "
Jul 27 18:13:18.042196 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.042112    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-serving-cert\") pod \"af864fee-581d-43c8-ae03-6a9a04e8a577\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") "
Jul 27 18:13:18.042196 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.042138    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-trusted-ca-bundle\") pod \"af864fee-581d-43c8-ae03-6a9a04e8a577\" (UID: \"af864fee-581d-43c8-ae03-6a9a04e8a577\") "
Jul 27 18:13:18.042751 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.042593    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-service-ca" (OuterVolumeSpecName: "service-ca") pod "af864fee-581d-43c8-ae03-6a9a04e8a577" (UID: "af864fee-581d-43c8-ae03-6a9a04e8a577"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:13:18.042875 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.042845    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "af864fee-581d-43c8-ae03-6a9a04e8a577" (UID: "af864fee-581d-43c8-ae03-6a9a04e8a577"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:13:18.042950 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.042863    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "af864fee-581d-43c8-ae03-6a9a04e8a577" (UID: "af864fee-581d-43c8-ae03-6a9a04e8a577"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:13:18.043161 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.043135    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-console-config" (OuterVolumeSpecName: "console-config") pod "af864fee-581d-43c8-ae03-6a9a04e8a577" (UID: "af864fee-581d-43c8-ae03-6a9a04e8a577"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:13:18.044497 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.044472    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "af864fee-581d-43c8-ae03-6a9a04e8a577" (UID: "af864fee-581d-43c8-ae03-6a9a04e8a577"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:13:18.044589 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.044570    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "af864fee-581d-43c8-ae03-6a9a04e8a577" (UID: "af864fee-581d-43c8-ae03-6a9a04e8a577"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:13:18.044789 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.044772    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af864fee-581d-43c8-ae03-6a9a04e8a577-kube-api-access-76qcl" (OuterVolumeSpecName: "kube-api-access-76qcl") pod "af864fee-581d-43c8-ae03-6a9a04e8a577" (UID: "af864fee-581d-43c8-ae03-6a9a04e8a577"). InnerVolumeSpecName "kube-api-access-76qcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:13:18.144027 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.143989    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76qcl\" (UniqueName: \"kubernetes.io/projected/af864fee-581d-43c8-ae03-6a9a04e8a577-kube-api-access-76qcl\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:18.144027 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.144023    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:18.144027 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.144032    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-console-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:18.144351 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.144040    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-oauth-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:18.144351 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.144051    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:18.144351 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.144060    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af864fee-581d-43c8-ae03-6a9a04e8a577-trusted-ca-bundle\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:18.144351 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.144068    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af864fee-581d-43c8-ae03-6a9a04e8a577-console-oauth-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:18.797125 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.797092    2584 generic.go:356] "Generic (PLEG): container finished" podID="af864fee-581d-43c8-ae03-6a9a04e8a577" containerID="b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51" exitCode=0
Jul 27 18:13:18.797610 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.797136    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b95f967bb-ptmvb"
Jul 27 18:13:18.797610 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.797177    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b95f967bb-ptmvb" event={"ID":"af864fee-581d-43c8-ae03-6a9a04e8a577","Type":"ContainerDied","Data":"b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51"}
Jul 27 18:13:18.797610 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.797239    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b95f967bb-ptmvb" event={"ID":"af864fee-581d-43c8-ae03-6a9a04e8a577","Type":"ContainerDied","Data":"5813a473bc89b5e6708ec7ad639f6687a2d19a1869c7b9e1aae66b26abe96290"}
Jul 27 18:13:18.797610 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.797274    2584 scope.go:117] "RemoveContainer" containerID="b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51"
Jul 27 18:13:18.807183 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.807165    2584 scope.go:117] "RemoveContainer" containerID="b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51"
Jul 27 18:13:18.807502 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:13:18.807481    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51\": container with ID starting with b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51 not found: ID does not exist" containerID="b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51"
Jul 27 18:13:18.807572 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.807510    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51"} err="failed to get container status \"b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51\": rpc error: code = NotFound desc = could not find container \"b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51\": container with ID starting with b42fc741c3738926758e148277a496f19764529d122f95d05daf3fba50e9ca51 not found: ID does not exist"
Jul 27 18:13:18.824415 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.824381    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b95f967bb-ptmvb"]
Jul 27 18:13:18.824576 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:18.824451    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b95f967bb-ptmvb"]
Jul 27 18:13:19.213232 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:19.213193    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:19.314917 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:19.314883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:20.779318 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:20.779282    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:20.824196 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:20.824163    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af864fee-581d-43c8-ae03-6a9a04e8a577" path="/var/lib/kubelet/pods/af864fee-581d-43c8-ae03-6a9a04e8a577/volumes"
Jul 27 18:13:20.879248 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:20.879214    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:29.015169 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.015076    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd_bbb1321c-2dff-4612-b2b0-d829ab6db438/payload-processing/0.log"
Jul 27 18:13:29.285942 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.285861    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx_5f39721d-3a00-4c50-a2db-c666f8e51003/payload-processing/0.log"
Jul 27 18:13:29.611448 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.611415    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:29.877821 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.877721    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"]
Jul 27 18:13:29.878306 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.878285    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af864fee-581d-43c8-ae03-6a9a04e8a577" containerName="console"
Jul 27 18:13:29.878408 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.878307    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="af864fee-581d-43c8-ae03-6a9a04e8a577" containerName="console"
Jul 27 18:13:29.878464 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.878424    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="af864fee-581d-43c8-ae03-6a9a04e8a577" containerName="console"
Jul 27 18:13:29.883060 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.883037    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:29.886509 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.886473    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-e962ab-openshift-default-dockercfg-54mbf\""
Jul 27 18:13:29.904001 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.903970    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"]
Jul 27 18:13:29.929396 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.929336    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:29.971516 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.970984    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-data\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:29.971789 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.971555    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfs7g\" (UniqueName: \"kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-kube-api-access-kfs7g\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:29.971875 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.971852    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-certs\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:29.971950 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.971934    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-token\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:29.972017 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.972003    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-envoy\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:29.972081 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.972033    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1d6f45-055e-4716-9d75-44270fd85b6c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:29.972145 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.972092    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-podinfo\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:29.972145 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.972135    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-socket\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:29.972396 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:29.972375    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-credential-socket\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.075216 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.075170    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-envoy\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.075216 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.075220    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1d6f45-055e-4716-9d75-44270fd85b6c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.075830 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.075261    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-podinfo\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.075830 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.075289    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-socket\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.075830 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.075318    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-credential-socket\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.075830 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.075405    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-data\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.075830 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.075433    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfs7g\" (UniqueName: \"kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-kube-api-access-kfs7g\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.075830 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.075478    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-certs\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.075830 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.075551    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-token\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.076212 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.075840    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-socket\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.076212 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.076153    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-certs\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.076406 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.076380    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-data\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.076471 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.076414    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1d6f45-055e-4716-9d75-44270fd85b6c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.076645 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.076621    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-credential-socket\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.078048 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.078017    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-envoy\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.078171 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.078136    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-podinfo\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.085104 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.085072    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-token\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.085460 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.085443    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfs7g\" (UniqueName: \"kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-kube-api-access-kfs7g\") pod \"e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") " pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.113291 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.113258    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:30.194218 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.194118    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:30.355170 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.355137    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"]
Jul 27 18:13:30.357364 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:13:30.357333    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1d6f45_055e_4716_9d75_44270fd85b6c.slice/crio-cace08f57144bc4d3a9f6e2efff8a418c9ecb8840e64851a56f0fc4c597857ac WatchSource:0}: Error finding container cace08f57144bc4d3a9f6e2efff8a418c9ecb8840e64851a56f0fc4c597857ac: Status 404 returned error can't find the container with id cace08f57144bc4d3a9f6e2efff8a418c9ecb8840e64851a56f0fc4c597857ac
Jul 27 18:13:30.359916 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.359870    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:13:30.360002 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.359971    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:13:30.360041 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.360023    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:13:30.803876 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.803836    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c64ccc968-j5t26" podUID="87c5bc5d-4716-4208-9c07-e558a94dd7ad" containerName="console" containerID="cri-o://5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91" gracePeriod=15
Jul 27 18:13:30.865716 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.865659    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" event={"ID":"ce1d6f45-055e-4716-9d75-44270fd85b6c","Type":"ContainerStarted","Data":"3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a"}
Jul 27 18:13:30.865919 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.865734    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" event={"ID":"ce1d6f45-055e-4716-9d75-44270fd85b6c","Type":"ContainerStarted","Data":"cace08f57144bc4d3a9f6e2efff8a418c9ecb8840e64851a56f0fc4c597857ac"}
Jul 27 18:13:30.890918 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:30.890859    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" podStartSLOduration=1.89084148 podStartE2EDuration="1.89084148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:13:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:13:30.890104183 +0000 UTC m=+3148.598252552" watchObservedRunningTime="2026-07-27 18:13:30.89084148 +0000 UTC m=+3148.598989798"
Jul 27 18:13:31.055767 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.055703    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:13:31.187574 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.187536    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4k5v\" (UniqueName: \"kubernetes.io/projected/87c5bc5d-4716-4208-9c07-e558a94dd7ad-kube-api-access-f4k5v\") pod \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") "
Jul 27 18:13:31.188091 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.187633    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-config\") pod \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") "
Jul 27 18:13:31.188091 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.187669    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-oauth-serving-cert\") pod \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") "
Jul 27 18:13:31.188091 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.187805    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-serving-cert\") pod \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") "
Jul 27 18:13:31.188091 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.187866    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-service-ca\") pod \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") "
Jul 27 18:13:31.188091 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.187909    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-trusted-ca-bundle\") pod \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") "
Jul 27 18:13:31.188091 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.187994    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-oauth-config\") pod \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\" (UID: \"87c5bc5d-4716-4208-9c07-e558a94dd7ad\") "
Jul 27 18:13:31.188091 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.188022    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-config" (OuterVolumeSpecName: "console-config") pod "87c5bc5d-4716-4208-9c07-e558a94dd7ad" (UID: "87c5bc5d-4716-4208-9c07-e558a94dd7ad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:13:31.188091 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.188050    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "87c5bc5d-4716-4208-9c07-e558a94dd7ad" (UID: "87c5bc5d-4716-4208-9c07-e558a94dd7ad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:13:31.188488 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.188270    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-service-ca" (OuterVolumeSpecName: "service-ca") pod "87c5bc5d-4716-4208-9c07-e558a94dd7ad" (UID: "87c5bc5d-4716-4208-9c07-e558a94dd7ad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:13:31.188488 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.188438    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:31.188488 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.188451    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-oauth-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:31.188488 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.188460    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:31.188812 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.188786    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "87c5bc5d-4716-4208-9c07-e558a94dd7ad" (UID: "87c5bc5d-4716-4208-9c07-e558a94dd7ad"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:13:31.190049 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.190020    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c5bc5d-4716-4208-9c07-e558a94dd7ad-kube-api-access-f4k5v" (OuterVolumeSpecName: "kube-api-access-f4k5v") pod "87c5bc5d-4716-4208-9c07-e558a94dd7ad" (UID: "87c5bc5d-4716-4208-9c07-e558a94dd7ad"). InnerVolumeSpecName "kube-api-access-f4k5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:13:31.190146 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.190063    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "87c5bc5d-4716-4208-9c07-e558a94dd7ad" (UID: "87c5bc5d-4716-4208-9c07-e558a94dd7ad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:13:31.190432 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.190413    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "87c5bc5d-4716-4208-9c07-e558a94dd7ad" (UID: "87c5bc5d-4716-4208-9c07-e558a94dd7ad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:13:31.194611 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.194593    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:31.200024 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.200001    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:31.289786 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.289755    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-serving-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:31.289786 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.289783    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87c5bc5d-4716-4208-9c07-e558a94dd7ad-trusted-ca-bundle\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:31.289786 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.289795    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87c5bc5d-4716-4208-9c07-e558a94dd7ad-console-oauth-config\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:31.290021 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.289805    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4k5v\" (UniqueName: \"kubernetes.io/projected/87c5bc5d-4716-4208-9c07-e558a94dd7ad-kube-api-access-f4k5v\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:13:31.876537 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.876509    2584 generic.go:356] "Generic (PLEG): container finished" podID="87c5bc5d-4716-4208-9c07-e558a94dd7ad" containerID="5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91" exitCode=0
Jul 27 18:13:31.876772 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.876559    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c64ccc968-j5t26"
Jul 27 18:13:31.876772 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.876594    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c64ccc968-j5t26" event={"ID":"87c5bc5d-4716-4208-9c07-e558a94dd7ad","Type":"ContainerDied","Data":"5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91"}
Jul 27 18:13:31.876772 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.876658    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c64ccc968-j5t26" event={"ID":"87c5bc5d-4716-4208-9c07-e558a94dd7ad","Type":"ContainerDied","Data":"b28f61cd43c4f9f2661230e67e903c41d13755f7a91b5b78e492ed0a3f04c4ac"}
Jul 27 18:13:31.876936 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.876781    2584 scope.go:117] "RemoveContainer" containerID="5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91"
Jul 27 18:13:31.876936 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.876921    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:31.878277 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.878222    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:31.887558 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.887540    2584 scope.go:117] "RemoveContainer" containerID="5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91"
Jul 27 18:13:31.887900 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:13:31.887877    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91\": container with ID starting with 5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91 not found: ID does not exist" containerID="5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91"
Jul 27 18:13:31.887970 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.887910    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91"} err="failed to get container status \"5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91\": rpc error: code = NotFound desc = could not find container \"5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91\": container with ID starting with 5ad88c7ba90c9a4de7c3ef5eca0f63ee45433e2ad00696209ede665d01e16b91 not found: ID does not exist"
Jul 27 18:13:31.920189 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.920152    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c64ccc968-j5t26"]
Jul 27 18:13:31.926320 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:31.926290    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c64ccc968-j5t26"]
Jul 27 18:13:32.825927 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:32.825895    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87c5bc5d-4716-4208-9c07-e558a94dd7ad" path="/var/lib/kubelet/pods/87c5bc5d-4716-4208-9c07-e558a94dd7ad/volumes"
Jul 27 18:13:32.836084 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:32.836055    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:34.591236 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:34.591202    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:34.991524 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:34.991434    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"]
Jul 27 18:13:34.992039 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:34.992020    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c5bc5d-4716-4208-9c07-e558a94dd7ad" containerName="console"
Jul 27 18:13:34.992151 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:34.992041    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c5bc5d-4716-4208-9c07-e558a94dd7ad" containerName="console"
Jul 27 18:13:34.992151 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:34.992143    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="87c5bc5d-4716-4208-9c07-e558a94dd7ad" containerName="console"
Jul 27 18:13:34.996794 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:34.996769    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:34.999731 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:34.999707    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-e962ab-serving-cert\""
Jul 27 18:13:35.004600 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.003987    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"]
Jul 27 18:13:35.019443 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.019405    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:35.133028 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.132994    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkbr2\" (UniqueName: \"kubernetes.io/projected/68786b2a-5f2b-4f17-8ceb-d32053f0def7-kube-api-access-dkbr2\") pod \"maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:35.133216 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.133066    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68786b2a-5f2b-4f17-8ceb-d32053f0def7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:35.133216 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.133200    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68786b2a-5f2b-4f17-8ceb-d32053f0def7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:35.173806 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.173767    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:35.234461 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.234424    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkbr2\" (UniqueName: \"kubernetes.io/projected/68786b2a-5f2b-4f17-8ceb-d32053f0def7-kube-api-access-dkbr2\") pod \"maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:35.234654 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.234512    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68786b2a-5f2b-4f17-8ceb-d32053f0def7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:35.234654 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.234596    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68786b2a-5f2b-4f17-8ceb-d32053f0def7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:35.235303 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.235278    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68786b2a-5f2b-4f17-8ceb-d32053f0def7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:35.237320 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.237295    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68786b2a-5f2b-4f17-8ceb-d32053f0def7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:35.243985 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.243914    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkbr2\" (UniqueName: \"kubernetes.io/projected/68786b2a-5f2b-4f17-8ceb-d32053f0def7-kube-api-access-dkbr2\") pod \"maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:35.309816 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.309776    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:35.459939 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.459914    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"]
Jul 27 18:13:35.462325 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:13:35.462291    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68786b2a_5f2b_4f17_8ceb_d32053f0def7.slice/crio-ca9ceb8f609062713b2d8374df32ee41c383fe48aa939039edcbb62384c32ca8 WatchSource:0}: Error finding container ca9ceb8f609062713b2d8374df32ee41c383fe48aa939039edcbb62384c32ca8: Status 404 returned error can't find the container with id ca9ceb8f609062713b2d8374df32ee41c383fe48aa939039edcbb62384c32ca8
Jul 27 18:13:35.604957 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.604929    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:35.704275 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.704232    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:35.742626 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.742590    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"]
Jul 27 18:13:35.747498 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.747479    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:13:35.750664 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.750637    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-e962ab-dockercfg-pzrml\""
Jul 27 18:13:35.750801 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.750673    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-e962ab\""
Jul 27 18:13:35.756849 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.756824    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"]
Jul 27 18:13:35.841436 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.841387    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88ced35-a900-46e3-b140-759db37bd087-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964\" (UID: \"a88ced35-a900-46e3-b140-759db37bd087\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:13:35.841637 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.841560    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77ql9\" (UniqueName: \"kubernetes.io/projected/a88ced35-a900-46e3-b140-759db37bd087-kube-api-access-77ql9\") pod \"payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964\" (UID: \"a88ced35-a900-46e3-b140-759db37bd087\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:13:35.899808 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.899724    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf" event={"ID":"68786b2a-5f2b-4f17-8ceb-d32053f0def7","Type":"ContainerStarted","Data":"ca9ceb8f609062713b2d8374df32ee41c383fe48aa939039edcbb62384c32ca8"}
Jul 27 18:13:35.942452 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.942412    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88ced35-a900-46e3-b140-759db37bd087-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964\" (UID: \"a88ced35-a900-46e3-b140-759db37bd087\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:13:35.942603 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.942520    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77ql9\" (UniqueName: \"kubernetes.io/projected/a88ced35-a900-46e3-b140-759db37bd087-kube-api-access-77ql9\") pod \"payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964\" (UID: \"a88ced35-a900-46e3-b140-759db37bd087\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:13:35.943216 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.943195    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88ced35-a900-46e3-b140-759db37bd087-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964\" (UID: \"a88ced35-a900-46e3-b140-759db37bd087\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:13:35.952111 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:35.952088    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77ql9\" (UniqueName: \"kubernetes.io/projected/a88ced35-a900-46e3-b140-759db37bd087-kube-api-access-77ql9\") pod \"payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964\" (UID: \"a88ced35-a900-46e3-b140-759db37bd087\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:13:36.060105 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:36.060079    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:13:36.209568 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:36.209540    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"]
Jul 27 18:13:36.211874 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:13:36.211850    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88ced35_a900_46e3_b140_759db37bd087.slice/crio-a77b70cdd17350fb4e88053cef654bbf9252ae77dac54cfcc56254995ac23e90 WatchSource:0}: Error finding container a77b70cdd17350fb4e88053cef654bbf9252ae77dac54cfcc56254995ac23e90: Status 404 returned error can't find the container with id a77b70cdd17350fb4e88053cef654bbf9252ae77dac54cfcc56254995ac23e90
Jul 27 18:13:36.910616 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:36.910584    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf" event={"ID":"68786b2a-5f2b-4f17-8ceb-d32053f0def7","Type":"ContainerStarted","Data":"8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce"}
Jul 27 18:13:36.911162 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:36.910727    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:36.912352 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:36.912319    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964" event={"ID":"a88ced35-a900-46e3-b140-759db37bd087","Type":"ContainerStarted","Data":"bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9"}
Jul 27 18:13:36.912352 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:36.912356    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964" event={"ID":"a88ced35-a900-46e3-b140-759db37bd087","Type":"ContainerStarted","Data":"a77b70cdd17350fb4e88053cef654bbf9252ae77dac54cfcc56254995ac23e90"}
Jul 27 18:13:36.912519 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:36.912383    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:13:36.931582 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:36.931529    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf" podStartSLOduration=2.342534532 podStartE2EDuration="2.931511645s" totalImagesPullingTime="588.977113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:13:36.930855176 +0000 UTC m=+3154.639003486" watchObservedRunningTime="2026-07-27 18:13:36.931511645 +0000 UTC m=+3154.639659990"
Jul 27 18:13:36.949884 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:36.949830    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964" podStartSLOduration=1.9498122740000001 podStartE2EDuration="1.949812274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:13:36.949057217 +0000 UTC m=+3154.657205537" watchObservedRunningTime="2026-07-27 18:13:36.949812274 +0000 UTC m=+3154.657960592"
Jul 27 18:13:37.312083 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:37.312054    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:42.923739 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:42.923706    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:13:47.921212 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:47.921180    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:13:53.018290 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:53.018255    2584 generic.go:356] "Generic (PLEG): container finished" podID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerID="3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a" exitCode=0
Jul 27 18:13:53.018736 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:53.018318    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" event={"ID":"ce1d6f45-055e-4716-9d75-44270fd85b6c","Type":"ContainerDied","Data":"3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a"}
Jul 27 18:13:53.018802 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:53.018750    2584 scope.go:117] "RemoveContainer" containerID="3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a"
Jul 27 18:13:53.020099 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:53.020070    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:13:53.020170 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:53.020141    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:13:53.020206 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:53.020176    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 18:13:54.026767 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:54.026727    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" event={"ID":"ce1d6f45-055e-4716-9d75-44270fd85b6c","Type":"ContainerStarted","Data":"2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67"}
Jul 27 18:13:54.194971 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:54.194934    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:54.195350 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:54.195323    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 27 18:13:54.195412 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:54.195394    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 27 18:13:55.195447 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:55.195400    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 27 18:13:55.195919 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:55.195463    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 27 18:13:56.195259 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:56.195222    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 27 18:13:56.195442 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:56.195281    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 27 18:13:57.195543 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:57.195495    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 27 18:13:57.195987 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:57.195574    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 27 18:13:58.198784 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:58.198753    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:58.199177 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:58.199149    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:13:58.199775 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:13:58.199757    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:14:06.168760 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.168722    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"]
Jul 27 18:14:06.169217 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.168993    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf" podUID="68786b2a-5f2b-4f17-8ceb-d32053f0def7" containerName="maas-api" containerID="cri-o://8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce" gracePeriod=30
Jul 27 18:14:06.203386 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.203350    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:06.228167 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.228130    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"]
Jul 27 18:14:06.228636 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.228386    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964" podUID="a88ced35-a900-46e3-b140-759db37bd087" containerName="payload-processing" containerID="cri-o://bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9" gracePeriod=30
Jul 27 18:14:06.435627 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.435532    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:06.506278 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.506248    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:14:06.519964 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.519929    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:06.540438 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.540410    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:14:06.607864 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.607823    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68786b2a-5f2b-4f17-8ceb-d32053f0def7-openshift-service-ca\") pod \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") "
Jul 27 18:14:06.608088 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.607895    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkbr2\" (UniqueName: \"kubernetes.io/projected/68786b2a-5f2b-4f17-8ceb-d32053f0def7-kube-api-access-dkbr2\") pod \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") "
Jul 27 18:14:06.608088 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.607997    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88ced35-a900-46e3-b140-759db37bd087-plugins-config-volume\") pod \"a88ced35-a900-46e3-b140-759db37bd087\" (UID: \"a88ced35-a900-46e3-b140-759db37bd087\") "
Jul 27 18:14:06.608088 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.608041    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68786b2a-5f2b-4f17-8ceb-d32053f0def7-maas-api-tls\") pod \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\" (UID: \"68786b2a-5f2b-4f17-8ceb-d32053f0def7\") "
Jul 27 18:14:06.608248 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.608096    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77ql9\" (UniqueName: \"kubernetes.io/projected/a88ced35-a900-46e3-b140-759db37bd087-kube-api-access-77ql9\") pod \"a88ced35-a900-46e3-b140-759db37bd087\" (UID: \"a88ced35-a900-46e3-b140-759db37bd087\") "
Jul 27 18:14:06.608299 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.608262    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68786b2a-5f2b-4f17-8ceb-d32053f0def7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68786b2a-5f2b-4f17-8ceb-d32053f0def7" (UID: "68786b2a-5f2b-4f17-8ceb-d32053f0def7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:14:06.608419 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.608390    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a88ced35-a900-46e3-b140-759db37bd087-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a88ced35-a900-46e3-b140-759db37bd087" (UID: "a88ced35-a900-46e3-b140-759db37bd087"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:14:06.608577 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.608561    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68786b2a-5f2b-4f17-8ceb-d32053f0def7-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:06.608654 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.608580    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88ced35-a900-46e3-b140-759db37bd087-plugins-config-volume\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:06.610303 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.610277    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68786b2a-5f2b-4f17-8ceb-d32053f0def7-kube-api-access-dkbr2" (OuterVolumeSpecName: "kube-api-access-dkbr2") pod "68786b2a-5f2b-4f17-8ceb-d32053f0def7" (UID: "68786b2a-5f2b-4f17-8ceb-d32053f0def7"). InnerVolumeSpecName "kube-api-access-dkbr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:14:06.610303 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.610283    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68786b2a-5f2b-4f17-8ceb-d32053f0def7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68786b2a-5f2b-4f17-8ceb-d32053f0def7" (UID: "68786b2a-5f2b-4f17-8ceb-d32053f0def7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:14:06.610457 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.610318    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a88ced35-a900-46e3-b140-759db37bd087-kube-api-access-77ql9" (OuterVolumeSpecName: "kube-api-access-77ql9") pod "a88ced35-a900-46e3-b140-759db37bd087" (UID: "a88ced35-a900-46e3-b140-759db37bd087"). InnerVolumeSpecName "kube-api-access-77ql9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:14:06.709735 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.709625    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkbr2\" (UniqueName: \"kubernetes.io/projected/68786b2a-5f2b-4f17-8ceb-d32053f0def7-kube-api-access-dkbr2\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:06.709735 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.709658    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68786b2a-5f2b-4f17-8ceb-d32053f0def7-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:06.709735 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:06.709668    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77ql9\" (UniqueName: \"kubernetes.io/projected/a88ced35-a900-46e3-b140-759db37bd087-kube-api-access-77ql9\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:07.102458 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.102420    2584 generic.go:356] "Generic (PLEG): container finished" podID="68786b2a-5f2b-4f17-8ceb-d32053f0def7" containerID="8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce" exitCode=0
Jul 27 18:14:07.102649 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.102472    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"
Jul 27 18:14:07.102649 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.102501    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf" event={"ID":"68786b2a-5f2b-4f17-8ceb-d32053f0def7","Type":"ContainerDied","Data":"8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce"}
Jul 27 18:14:07.102649 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.102556    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf" event={"ID":"68786b2a-5f2b-4f17-8ceb-d32053f0def7","Type":"ContainerDied","Data":"ca9ceb8f609062713b2d8374df32ee41c383fe48aa939039edcbb62384c32ca8"}
Jul 27 18:14:07.102649 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.102580    2584 scope.go:117] "RemoveContainer" containerID="8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce"
Jul 27 18:14:07.104035 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.104013    2584 generic.go:356] "Generic (PLEG): container finished" podID="a88ced35-a900-46e3-b140-759db37bd087" containerID="bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9" exitCode=0
Jul 27 18:14:07.104137 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.104067    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"
Jul 27 18:14:07.104378 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.104065    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964" event={"ID":"a88ced35-a900-46e3-b140-759db37bd087","Type":"ContainerDied","Data":"bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9"}
Jul 27 18:14:07.104378 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.104179    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964" event={"ID":"a88ced35-a900-46e3-b140-759db37bd087","Type":"ContainerDied","Data":"a77b70cdd17350fb4e88053cef654bbf9252ae77dac54cfcc56254995ac23e90"}
Jul 27 18:14:07.115706 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.112623    2584 scope.go:117] "RemoveContainer" containerID="8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce"
Jul 27 18:14:07.115706 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:14:07.114797    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce\": container with ID starting with 8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce not found: ID does not exist" containerID="8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce"
Jul 27 18:14:07.115706 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.114845    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce"} err="failed to get container status \"8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce\": rpc error: code = NotFound desc = could not find container \"8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce\": container with ID starting with 8193114d78c96c684250d415cdd1b9ee663a270377c97e8e4c5244b3d86bafce not found: ID does not exist"
Jul 27 18:14:07.115706 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.114869    2584 scope.go:117] "RemoveContainer" containerID="bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9"
Jul 27 18:14:07.124963 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.124928    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"]
Jul 27 18:14:07.129472 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.129451    2584 scope.go:117] "RemoveContainer" containerID="bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9"
Jul 27 18:14:07.129552 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.129479    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e962ab-6b88dcbbf4-kz964"]
Jul 27 18:14:07.129804 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:14:07.129787    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9\": container with ID starting with bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9 not found: ID does not exist" containerID="bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9"
Jul 27 18:14:07.129854 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.129813    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9"} err="failed to get container status \"bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9\": rpc error: code = NotFound desc = could not find container \"bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9\": container with ID starting with bc5ff5ba1b9c17c46b33a95e04030f607677d24a2b4917da35a07e243891c3c9 not found: ID does not exist"
Jul 27 18:14:07.141925 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.141897    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"]
Jul 27 18:14:07.147935 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:07.147905    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e962ab-54bc87cdd-xtcsf"]
Jul 27 18:14:08.423222 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:08.423189    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:08.824067 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:08.824027    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68786b2a-5f2b-4f17-8ceb-d32053f0def7" path="/var/lib/kubelet/pods/68786b2a-5f2b-4f17-8ceb-d32053f0def7/volumes"
Jul 27 18:14:08.824520 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:08.824500    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a88ced35-a900-46e3-b140-759db37bd087" path="/var/lib/kubelet/pods/a88ced35-a900-46e3-b140-759db37bd087/volumes"
Jul 27 18:14:08.977008 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:08.976970    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:14.829657 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:14.829628    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:14.934771 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:14.934731    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:15.080556 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:15.080461    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"]
Jul 27 18:14:15.080800 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:15.080773    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerName="istio-proxy" containerID="cri-o://2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67" gracePeriod=30
Jul 27 18:14:15.101762 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:15.101729    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:15.319010 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:15.318963    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:20.349361 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.349293    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:14:20.354912 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.354888    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-token\") pod \"ce1d6f45-055e-4716-9d75-44270fd85b6c\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") "
Jul 27 18:14:20.355055 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.354944    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-certs\") pod \"ce1d6f45-055e-4716-9d75-44270fd85b6c\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") "
Jul 27 18:14:20.355055 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.354973    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-podinfo\") pod \"ce1d6f45-055e-4716-9d75-44270fd85b6c\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") "
Jul 27 18:14:20.355055 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.355049    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-credential-socket\") pod \"ce1d6f45-055e-4716-9d75-44270fd85b6c\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") "
Jul 27 18:14:20.355171 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.355077    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1d6f45-055e-4716-9d75-44270fd85b6c-istiod-ca-cert\") pod \"ce1d6f45-055e-4716-9d75-44270fd85b6c\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") "
Jul 27 18:14:20.355171 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.355131    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfs7g\" (UniqueName: \"kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-kube-api-access-kfs7g\") pod \"ce1d6f45-055e-4716-9d75-44270fd85b6c\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") "
Jul 27 18:14:20.355257 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.355202    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-data\") pod \"ce1d6f45-055e-4716-9d75-44270fd85b6c\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") "
Jul 27 18:14:20.355257 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.355223    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-envoy\") pod \"ce1d6f45-055e-4716-9d75-44270fd85b6c\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") "
Jul 27 18:14:20.355257 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.355240    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ce1d6f45-055e-4716-9d75-44270fd85b6c" (UID: "ce1d6f45-055e-4716-9d75-44270fd85b6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:14:20.355401 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.355278    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-socket\") pod \"ce1d6f45-055e-4716-9d75-44270fd85b6c\" (UID: \"ce1d6f45-055e-4716-9d75-44270fd85b6c\") "
Jul 27 18:14:20.355799 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.355736    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-certs\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:20.355930 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.355801    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce1d6f45-055e-4716-9d75-44270fd85b6c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ce1d6f45-055e-4716-9d75-44270fd85b6c" (UID: "ce1d6f45-055e-4716-9d75-44270fd85b6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:14:20.355930 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.355896    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ce1d6f45-055e-4716-9d75-44270fd85b6c" (UID: "ce1d6f45-055e-4716-9d75-44270fd85b6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:14:20.356236 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.356207    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ce1d6f45-055e-4716-9d75-44270fd85b6c" (UID: "ce1d6f45-055e-4716-9d75-44270fd85b6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:14:20.356508 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.356471    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-data" (OuterVolumeSpecName: "istio-data") pod "ce1d6f45-055e-4716-9d75-44270fd85b6c" (UID: "ce1d6f45-055e-4716-9d75-44270fd85b6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:14:20.358220 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.358186    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ce1d6f45-055e-4716-9d75-44270fd85b6c" (UID: "ce1d6f45-055e-4716-9d75-44270fd85b6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:14:20.358333 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.358217    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ce1d6f45-055e-4716-9d75-44270fd85b6c" (UID: "ce1d6f45-055e-4716-9d75-44270fd85b6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 18:14:20.358549 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.358523    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-token" (OuterVolumeSpecName: "istio-token") pod "ce1d6f45-055e-4716-9d75-44270fd85b6c" (UID: "ce1d6f45-055e-4716-9d75-44270fd85b6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:14:20.358875 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.358846    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-kube-api-access-kfs7g" (OuterVolumeSpecName: "kube-api-access-kfs7g") pod "ce1d6f45-055e-4716-9d75-44270fd85b6c" (UID: "ce1d6f45-055e-4716-9d75-44270fd85b6c"). InnerVolumeSpecName "kube-api-access-kfs7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:14:20.456815 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.456782    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-data\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:20.456815 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.456810    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-envoy\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:20.456815 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.456819    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-workload-socket\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:20.457056 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.456827    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-token\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:20.457056 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.456836    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1d6f45-055e-4716-9d75-44270fd85b6c-istio-podinfo\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:20.457056 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.456844    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1d6f45-055e-4716-9d75-44270fd85b6c-credential-socket\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:20.457056 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.456851    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1d6f45-055e-4716-9d75-44270fd85b6c-istiod-ca-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:20.457056 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:20.456861    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfs7g\" (UniqueName: \"kubernetes.io/projected/ce1d6f45-055e-4716-9d75-44270fd85b6c-kube-api-access-kfs7g\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:21.181401 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.181304    2584 generic.go:356] "Generic (PLEG): container finished" podID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerID="2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67" exitCode=0
Jul 27 18:14:21.181401 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.181354    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"
Jul 27 18:14:21.181636 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.181397    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" event={"ID":"ce1d6f45-055e-4716-9d75-44270fd85b6c","Type":"ContainerDied","Data":"2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67"}
Jul 27 18:14:21.181636 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.181454    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p" event={"ID":"ce1d6f45-055e-4716-9d75-44270fd85b6c","Type":"ContainerDied","Data":"cace08f57144bc4d3a9f6e2efff8a418c9ecb8840e64851a56f0fc4c597857ac"}
Jul 27 18:14:21.181636 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.181470    2584 scope.go:117] "RemoveContainer" containerID="2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67"
Jul 27 18:14:21.191412 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.191393    2584 scope.go:117] "RemoveContainer" containerID="3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a"
Jul 27 18:14:21.199754 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.199725    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"]
Jul 27 18:14:21.200603 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.200437    2584 scope.go:117] "RemoveContainer" containerID="2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67"
Jul 27 18:14:21.200992 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:14:21.200964    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67\": container with ID starting with 2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67 not found: ID does not exist" containerID="2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67"
Jul 27 18:14:21.201112 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.201017    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67"} err="failed to get container status \"2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67\": rpc error: code = NotFound desc = could not find container \"2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67\": container with ID starting with 2dedabc2d58b47de25ee16ebccadc59af8671ab80158d87a47bf9e388d33fa67 not found: ID does not exist"
Jul 27 18:14:21.201112 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.201043    2584 scope.go:117] "RemoveContainer" containerID="3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a"
Jul 27 18:14:21.201403 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:14:21.201383    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a\": container with ID starting with 3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a not found: ID does not exist" containerID="3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a"
Jul 27 18:14:21.201467 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.201413    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a"} err="failed to get container status \"3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a\": rpc error: code = NotFound desc = could not find container \"3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a\": container with ID starting with 3a62e2c9e2738b788898c08ee96b12219b58dd8c043c4f37d7e076f52ef3135a not found: ID does not exist"
Jul 27 18:14:21.203603 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:21.203584    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e962ab-openshift-default-59679dffbb-hk56p"]
Jul 27 18:14:22.823708 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:22.823653    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" path="/var/lib/kubelet/pods/ce1d6f45-055e-4716-9d75-44270fd85b6c/volumes"
Jul 27 18:14:36.286031 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.285997    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:36.314495 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.314459    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"]
Jul 27 18:14:36.314803 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.314752    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd" podUID="bbb1321c-2dff-4612-b2b0-d829ab6db438" containerName="payload-processing" containerID="cri-o://fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3" gracePeriod=30
Jul 27 18:14:36.322198 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.322167    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"]
Jul 27 18:14:36.322440 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.322411    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj" podUID="c5bdb232-185b-4963-b0a2-099c4549af19" containerName="payload-pre-processing" containerID="cri-o://b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35" gracePeriod=30
Jul 27 18:14:36.509976 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.509938    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:36.570591 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.570557    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:36.603323 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.603295    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:14:36.630585 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.630553    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:14:36.728617 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.728579    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmqdr\" (UniqueName: \"kubernetes.io/projected/bbb1321c-2dff-4612-b2b0-d829ab6db438-kube-api-access-jmqdr\") pod \"bbb1321c-2dff-4612-b2b0-d829ab6db438\" (UID: \"bbb1321c-2dff-4612-b2b0-d829ab6db438\") "
Jul 27 18:14:36.728827 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.728627    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbb1321c-2dff-4612-b2b0-d829ab6db438-plugins-config-volume\") pod \"bbb1321c-2dff-4612-b2b0-d829ab6db438\" (UID: \"bbb1321c-2dff-4612-b2b0-d829ab6db438\") "
Jul 27 18:14:36.728827 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.728662    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg9lw\" (UniqueName: \"kubernetes.io/projected/c5bdb232-185b-4963-b0a2-099c4549af19-kube-api-access-sg9lw\") pod \"c5bdb232-185b-4963-b0a2-099c4549af19\" (UID: \"c5bdb232-185b-4963-b0a2-099c4549af19\") "
Jul 27 18:14:36.728827 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.728681    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bdb232-185b-4963-b0a2-099c4549af19-plugins-config-volume\") pod \"c5bdb232-185b-4963-b0a2-099c4549af19\" (UID: \"c5bdb232-185b-4963-b0a2-099c4549af19\") "
Jul 27 18:14:36.729138 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.729107    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbb1321c-2dff-4612-b2b0-d829ab6db438-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bbb1321c-2dff-4612-b2b0-d829ab6db438" (UID: "bbb1321c-2dff-4612-b2b0-d829ab6db438"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:14:36.729138 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.729130    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5bdb232-185b-4963-b0a2-099c4549af19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c5bdb232-185b-4963-b0a2-099c4549af19" (UID: "c5bdb232-185b-4963-b0a2-099c4549af19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:14:36.731021 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.730996    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbb1321c-2dff-4612-b2b0-d829ab6db438-kube-api-access-jmqdr" (OuterVolumeSpecName: "kube-api-access-jmqdr") pod "bbb1321c-2dff-4612-b2b0-d829ab6db438" (UID: "bbb1321c-2dff-4612-b2b0-d829ab6db438"). InnerVolumeSpecName "kube-api-access-jmqdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:14:36.731113 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.731063    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5bdb232-185b-4963-b0a2-099c4549af19-kube-api-access-sg9lw" (OuterVolumeSpecName: "kube-api-access-sg9lw") pod "c5bdb232-185b-4963-b0a2-099c4549af19" (UID: "c5bdb232-185b-4963-b0a2-099c4549af19"). InnerVolumeSpecName "kube-api-access-sg9lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:14:36.829778 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.829746    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmqdr\" (UniqueName: \"kubernetes.io/projected/bbb1321c-2dff-4612-b2b0-d829ab6db438-kube-api-access-jmqdr\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:36.829778 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.829775    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbb1321c-2dff-4612-b2b0-d829ab6db438-plugins-config-volume\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:36.829778 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.829784    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg9lw\" (UniqueName: \"kubernetes.io/projected/c5bdb232-185b-4963-b0a2-099c4549af19-kube-api-access-sg9lw\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:36.830038 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:36.829792    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bdb232-185b-4963-b0a2-099c4549af19-plugins-config-volume\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:14:37.271585 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.271492    2584 generic.go:356] "Generic (PLEG): container finished" podID="bbb1321c-2dff-4612-b2b0-d829ab6db438" containerID="fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3" exitCode=0
Jul 27 18:14:37.271585 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.271534    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"
Jul 27 18:14:37.271585 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.271578    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd" event={"ID":"bbb1321c-2dff-4612-b2b0-d829ab6db438","Type":"ContainerDied","Data":"fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3"}
Jul 27 18:14:37.271907 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.271633    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd" event={"ID":"bbb1321c-2dff-4612-b2b0-d829ab6db438","Type":"ContainerDied","Data":"5adfb9eb100ac731158f7e599c5c61ae3ef1068b172c98507f7768f608ef9064"}
Jul 27 18:14:37.271907 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.271663    2584 scope.go:117] "RemoveContainer" containerID="fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3"
Jul 27 18:14:37.273185 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.273165    2584 generic.go:356] "Generic (PLEG): container finished" podID="c5bdb232-185b-4963-b0a2-099c4549af19" containerID="b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35" exitCode=0
Jul 27 18:14:37.273286 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.273205    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj" event={"ID":"c5bdb232-185b-4963-b0a2-099c4549af19","Type":"ContainerDied","Data":"b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35"}
Jul 27 18:14:37.273286 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.273214    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"
Jul 27 18:14:37.273286 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.273244    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj" event={"ID":"c5bdb232-185b-4963-b0a2-099c4549af19","Type":"ContainerDied","Data":"2443962c8d72d1115f045d0e0c3103adf34301400f7f369d54d50c81b296c511"}
Jul 27 18:14:37.281721 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.281678    2584 scope.go:117] "RemoveContainer" containerID="fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3"
Jul 27 18:14:37.282019 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:14:37.281999    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3\": container with ID starting with fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3 not found: ID does not exist" containerID="fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3"
Jul 27 18:14:37.282078 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.282027    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3"} err="failed to get container status \"fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3\": rpc error: code = NotFound desc = could not find container \"fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3\": container with ID starting with fb129a340bee694286ec00d8e9563fdc5f46040b22ec8824f94fe86fdd11d4f3 not found: ID does not exist"
Jul 27 18:14:37.282078 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.282044    2584 scope.go:117] "RemoveContainer" containerID="b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35"
Jul 27 18:14:37.290635 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.290545    2584 scope.go:117] "RemoveContainer" containerID="b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35"
Jul 27 18:14:37.291020 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:14:37.290888    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35\": container with ID starting with b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35 not found: ID does not exist" containerID="b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35"
Jul 27 18:14:37.291020 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.290924    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35"} err="failed to get container status \"b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35\": rpc error: code = NotFound desc = could not find container \"b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35\": container with ID starting with b97e8d4205df7d82780126870ba2e6455f5b7734979313d09ef9be45b95eab35 not found: ID does not exist"
Jul 27 18:14:37.293318 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.293297    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"]
Jul 27 18:14:37.297013 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.296989    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7985e5-659c467c76-bhmjj"]
Jul 27 18:14:37.309950 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.309921    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"]
Jul 27 18:14:37.313955 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:37.313930    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7985e5-7588b449bd-pq7jd"]
Jul 27 18:14:38.691774 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:38.691741    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:38.823802 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:38.823759    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbb1321c-2dff-4612-b2b0-d829ab6db438" path="/var/lib/kubelet/pods/bbb1321c-2dff-4612-b2b0-d829ab6db438/volumes"
Jul 27 18:14:38.824174 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:38.824160    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5bdb232-185b-4963-b0a2-099c4549af19" path="/var/lib/kubelet/pods/c5bdb232-185b-4963-b0a2-099c4549af19/volumes"
Jul 27 18:14:39.072942 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:39.072907    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:44.135168 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:44.135127    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:14:50.511201 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:14:50.511166    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:23.045650 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:23.045612    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:23.109146 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:23.109104    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:27.929778 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:27.929748    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:28.033902 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:28.033861    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:28.213530 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:28.213445    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:28.489407 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:28.489310    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:39.018054 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.018017    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"]
Jul 27 18:15:39.018532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.018257    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5" podUID="963f1f41-6d40-4c97-a3dd-03745d101b6e" containerName="maas-api" containerID="cri-o://e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab" gracePeriod=30
Jul 27 18:15:39.049375 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.049333    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:39.070231 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.070182    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"]
Jul 27 18:15:39.070485 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.070455    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx" podUID="5f39721d-3a00-4c50-a2db-c666f8e51003" containerName="payload-processing" containerID="cri-o://21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028" gracePeriod=30
Jul 27 18:15:39.330109 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.330077    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:15:39.371871 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.371842    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:15:39.376749 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.376668    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963f1f41-6d40-4c97-a3dd-03745d101b6e-maas-api-tls\") pod \"963f1f41-6d40-4c97-a3dd-03745d101b6e\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") "
Jul 27 18:15:39.376904 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.376761    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963f1f41-6d40-4c97-a3dd-03745d101b6e-openshift-service-ca\") pod \"963f1f41-6d40-4c97-a3dd-03745d101b6e\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") "
Jul 27 18:15:39.376904 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.376859    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbt9g\" (UniqueName: \"kubernetes.io/projected/963f1f41-6d40-4c97-a3dd-03745d101b6e-kube-api-access-vbt9g\") pod \"963f1f41-6d40-4c97-a3dd-03745d101b6e\" (UID: \"963f1f41-6d40-4c97-a3dd-03745d101b6e\") "
Jul 27 18:15:39.377226 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.377197    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/963f1f41-6d40-4c97-a3dd-03745d101b6e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "963f1f41-6d40-4c97-a3dd-03745d101b6e" (UID: "963f1f41-6d40-4c97-a3dd-03745d101b6e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:15:39.379653 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.379625    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/963f1f41-6d40-4c97-a3dd-03745d101b6e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "963f1f41-6d40-4c97-a3dd-03745d101b6e" (UID: "963f1f41-6d40-4c97-a3dd-03745d101b6e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:15:39.379653 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.379645    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/963f1f41-6d40-4c97-a3dd-03745d101b6e-kube-api-access-vbt9g" (OuterVolumeSpecName: "kube-api-access-vbt9g") pod "963f1f41-6d40-4c97-a3dd-03745d101b6e" (UID: "963f1f41-6d40-4c97-a3dd-03745d101b6e"). InnerVolumeSpecName "kube-api-access-vbt9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:15:39.478179 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.478138    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f39721d-3a00-4c50-a2db-c666f8e51003-plugins-config-volume\") pod \"5f39721d-3a00-4c50-a2db-c666f8e51003\" (UID: \"5f39721d-3a00-4c50-a2db-c666f8e51003\") "
Jul 27 18:15:39.478391 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.478206    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkkzn\" (UniqueName: \"kubernetes.io/projected/5f39721d-3a00-4c50-a2db-c666f8e51003-kube-api-access-tkkzn\") pod \"5f39721d-3a00-4c50-a2db-c666f8e51003\" (UID: \"5f39721d-3a00-4c50-a2db-c666f8e51003\") "
Jul 27 18:15:39.478557 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.478534    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f39721d-3a00-4c50-a2db-c666f8e51003-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f39721d-3a00-4c50-a2db-c666f8e51003" (UID: "5f39721d-3a00-4c50-a2db-c666f8e51003"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:15:39.478625 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.478606    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbt9g\" (UniqueName: \"kubernetes.io/projected/963f1f41-6d40-4c97-a3dd-03745d101b6e-kube-api-access-vbt9g\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:15:39.478716 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.478628    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963f1f41-6d40-4c97-a3dd-03745d101b6e-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:15:39.478716 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.478644    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963f1f41-6d40-4c97-a3dd-03745d101b6e-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:15:39.480523 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.480500    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f39721d-3a00-4c50-a2db-c666f8e51003-kube-api-access-tkkzn" (OuterVolumeSpecName: "kube-api-access-tkkzn") pod "5f39721d-3a00-4c50-a2db-c666f8e51003" (UID: "5f39721d-3a00-4c50-a2db-c666f8e51003"). InnerVolumeSpecName "kube-api-access-tkkzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:15:39.579732 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.579673    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f39721d-3a00-4c50-a2db-c666f8e51003-plugins-config-volume\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:15:39.579732 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.579728    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkkzn\" (UniqueName: \"kubernetes.io/projected/5f39721d-3a00-4c50-a2db-c666f8e51003-kube-api-access-tkkzn\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:15:39.609364 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.609331    2584 generic.go:356] "Generic (PLEG): container finished" podID="5f39721d-3a00-4c50-a2db-c666f8e51003" containerID="21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028" exitCode=0
Jul 27 18:15:39.609553 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.609388    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"
Jul 27 18:15:39.609553 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.609413    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx" event={"ID":"5f39721d-3a00-4c50-a2db-c666f8e51003","Type":"ContainerDied","Data":"21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028"}
Jul 27 18:15:39.609553 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.609464    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx" event={"ID":"5f39721d-3a00-4c50-a2db-c666f8e51003","Type":"ContainerDied","Data":"0a97857bdbdf5e9ae859f1666602401daaa389261100a8ed5e499745eb6b8b30"}
Jul 27 18:15:39.609553 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.609486    2584 scope.go:117] "RemoveContainer" containerID="21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028"
Jul 27 18:15:39.610814 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.610795    2584 generic.go:356] "Generic (PLEG): container finished" podID="963f1f41-6d40-4c97-a3dd-03745d101b6e" containerID="e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab" exitCode=0
Jul 27 18:15:39.610901 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.610845    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"
Jul 27 18:15:39.610901 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.610871    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5" event={"ID":"963f1f41-6d40-4c97-a3dd-03745d101b6e","Type":"ContainerDied","Data":"e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab"}
Jul 27 18:15:39.610983 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.610906    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5" event={"ID":"963f1f41-6d40-4c97-a3dd-03745d101b6e","Type":"ContainerDied","Data":"2078e90b58f503575ad9a00eef499f507a044bfcc59365a759e562bbf8bd4c1c"}
Jul 27 18:15:39.620161 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.620142    2584 scope.go:117] "RemoveContainer" containerID="21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028"
Jul 27 18:15:39.620504 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:15:39.620478    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028\": container with ID starting with 21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028 not found: ID does not exist" containerID="21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028"
Jul 27 18:15:39.620583 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.620516    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028"} err="failed to get container status \"21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028\": rpc error: code = NotFound desc = could not find container \"21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028\": container with ID starting with 21273a9ffa6f637f755aede3b8080339eb6614b7ffdf81b5af002c34ea23b028 not found: ID does not exist"
Jul 27 18:15:39.620583 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.620534    2584 scope.go:117] "RemoveContainer" containerID="e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab"
Jul 27 18:15:39.629033 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.629014    2584 scope.go:117] "RemoveContainer" containerID="e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab"
Jul 27 18:15:39.629352 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:15:39.629327    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab\": container with ID starting with e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab not found: ID does not exist" containerID="e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab"
Jul 27 18:15:39.629440 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.629362    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab"} err="failed to get container status \"e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab\": rpc error: code = NotFound desc = could not find container \"e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab\": container with ID starting with e32d6f497d8153a0321d27be7eb8627e3b2f030da6359faa28a7e389043036ab not found: ID does not exist"
Jul 27 18:15:39.632875 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.632854    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"]
Jul 27 18:15:39.637626 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.637606    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fbef1a-6f4fc944b-mknzx"]
Jul 27 18:15:39.647583 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.647555    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"]
Jul 27 18:15:39.651313 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:39.651291    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-fbef1a-7bc8fcb646-n5pp5"]
Jul 27 18:15:40.214715 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:40.214661    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:40.313933 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:40.313902    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:40.824139 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:40.824102    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f39721d-3a00-4c50-a2db-c666f8e51003" path="/var/lib/kubelet/pods/5f39721d-3a00-4c50-a2db-c666f8e51003/volumes"
Jul 27 18:15:40.824511 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:40.824498    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="963f1f41-6d40-4c97-a3dd-03745d101b6e" path="/var/lib/kubelet/pods/963f1f41-6d40-4c97-a3dd-03745d101b6e/volumes"
Jul 27 18:15:43.648078 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:43.648044    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:44.032905 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:44.032823    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:50.427228 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:50.427176    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:50.532593 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:50.532539    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:50.695971 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:50.695853    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:15:50.904434 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:15:50.904392    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:03.322181 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.322143    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:03.363197 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.363145    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"]
Jul 27 18:16:03.363439 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.363406    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25" podUID="c0e4b412-5c96-42a4-948d-1b53f9622f98" containerName="payload-processing" containerID="cri-o://06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3" gracePeriod=30
Jul 27 18:16:03.402124 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.402093    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:03.464186 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.464102    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:03.632207 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.632175    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:16:03.732347 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.732256    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e4b412-5c96-42a4-948d-1b53f9622f98-plugins-config-volume\") pod \"c0e4b412-5c96-42a4-948d-1b53f9622f98\" (UID: \"c0e4b412-5c96-42a4-948d-1b53f9622f98\") "
Jul 27 18:16:03.732502 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.732381    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8zpb\" (UniqueName: \"kubernetes.io/projected/c0e4b412-5c96-42a4-948d-1b53f9622f98-kube-api-access-k8zpb\") pod \"c0e4b412-5c96-42a4-948d-1b53f9622f98\" (UID: \"c0e4b412-5c96-42a4-948d-1b53f9622f98\") "
Jul 27 18:16:03.732763 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.732723    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0e4b412-5c96-42a4-948d-1b53f9622f98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c0e4b412-5c96-42a4-948d-1b53f9622f98" (UID: "c0e4b412-5c96-42a4-948d-1b53f9622f98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:16:03.734614 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.734588    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e4b412-5c96-42a4-948d-1b53f9622f98-kube-api-access-k8zpb" (OuterVolumeSpecName: "kube-api-access-k8zpb") pod "c0e4b412-5c96-42a4-948d-1b53f9622f98" (UID: "c0e4b412-5c96-42a4-948d-1b53f9622f98"). InnerVolumeSpecName "kube-api-access-k8zpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:16:03.742611 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.742585    2584 generic.go:356] "Generic (PLEG): container finished" podID="c0e4b412-5c96-42a4-948d-1b53f9622f98" containerID="06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3" exitCode=0
Jul 27 18:16:03.742736 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.742641    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25" event={"ID":"c0e4b412-5c96-42a4-948d-1b53f9622f98","Type":"ContainerDied","Data":"06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3"}
Jul 27 18:16:03.742736 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.742647    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"
Jul 27 18:16:03.742736 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.742723    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25" event={"ID":"c0e4b412-5c96-42a4-948d-1b53f9622f98","Type":"ContainerDied","Data":"cf4780d5f001664f2ed94cf6c2ca8e3236839a32c12dba8c6e0f68e8c2f4e68f"}
Jul 27 18:16:03.742854 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.742746    2584 scope.go:117] "RemoveContainer" containerID="06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3"
Jul 27 18:16:03.751666 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.751648    2584 scope.go:117] "RemoveContainer" containerID="06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3"
Jul 27 18:16:03.751962 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:16:03.751945    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3\": container with ID starting with 06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3 not found: ID does not exist" containerID="06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3"
Jul 27 18:16:03.752013 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.751973    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3"} err="failed to get container status \"06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3\": rpc error: code = NotFound desc = could not find container \"06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3\": container with ID starting with 06dfcc308e39403b2d82618af47359538fb259f70604901b12384410b3ed0fc3 not found: ID does not exist"
Jul 27 18:16:03.765587 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.765555    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"]
Jul 27 18:16:03.768198 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.768160    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-1aa3ff-574d96d7d6-j2s25"]
Jul 27 18:16:03.834026 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.833982    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8zpb\" (UniqueName: \"kubernetes.io/projected/c0e4b412-5c96-42a4-948d-1b53f9622f98-kube-api-access-k8zpb\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:03.834026 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:03.834013    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e4b412-5c96-42a4-948d-1b53f9622f98-plugins-config-volume\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:04.824179 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:04.824143    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0e4b412-5c96-42a4-948d-1b53f9622f98" path="/var/lib/kubelet/pods/c0e4b412-5c96-42a4-948d-1b53f9622f98/volumes"
Jul 27 18:16:06.365104 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:06.365050    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:06.673004 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:06.672889    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:13.043563 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:13.043514    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:13.144673 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:13.144636    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:13.283421 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:13.283383    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"]
Jul 27 18:16:13.283652 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:13.283627    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" podUID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerName="istio-proxy" containerID="cri-o://b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8" gracePeriod=30
Jul 27 18:16:13.300416 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:13.300336    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:13.536090 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:13.536049    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:18.539142 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.539112    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:16:18.587021 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.586970    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istiod-ca-cert\") pod \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") "
Jul 27 18:16:18.587532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587041    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-envoy\") pod \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") "
Jul 27 18:16:18.587532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587155    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxlvs\" (UniqueName: \"kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-kube-api-access-qxlvs\") pod \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") "
Jul 27 18:16:18.587532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587211    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-data\") pod \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") "
Jul 27 18:16:18.587532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587255    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-certs\") pod \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") "
Jul 27 18:16:18.587532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587332    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-podinfo\") pod \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") "
Jul 27 18:16:18.587532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587360    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-credential-socket\") pod \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") "
Jul 27 18:16:18.587532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587423    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-socket\") pod \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") "
Jul 27 18:16:18.587532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587460    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-token\") pod \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\" (UID: \"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35\") "
Jul 27 18:16:18.587532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587422    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" (UID: "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:16:18.588048 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587531    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" (UID: "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:16:18.588048 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587744    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" (UID: "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:16:18.588048 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.587895    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istiod-ca-cert\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:18.588223 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.588068    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" (UID: "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:16:18.588223 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.588106    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-certs\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:18.588223 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.588104    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" (UID: "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:16:18.588223 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.588124    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-credential-socket\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:18.589735 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.589680    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-kube-api-access-qxlvs" (OuterVolumeSpecName: "kube-api-access-qxlvs") pod "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" (UID: "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35"). InnerVolumeSpecName "kube-api-access-qxlvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:16:18.589985 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.589960    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" (UID: "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:16:18.589985 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.589984    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" (UID: "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:16:18.590176 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.590049    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" (UID: "7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 18:16:18.689216 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.689176    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-envoy\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:18.689216 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.689212    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxlvs\" (UniqueName: \"kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-kube-api-access-qxlvs\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:18.689216 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.689222    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-data\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:18.689468 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.689232    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-podinfo\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:18.689468 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.689240    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-workload-socket\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:18.689468 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.689248    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35-istio-token\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:18.823546 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.823513    2584 generic.go:356] "Generic (PLEG): container finished" podID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerID="b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8" exitCode=0
Jul 27 18:16:18.823726 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.823623    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"
Jul 27 18:16:18.823856 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.823834    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" event={"ID":"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35","Type":"ContainerDied","Data":"b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8"}
Jul 27 18:16:18.823910 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.823879    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s" event={"ID":"7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35","Type":"ContainerDied","Data":"d89ff397cabc7d42457ebbecc0d88a36ef2a8064ad3fca311d3e7bf0c7bbf7cb"}
Jul 27 18:16:18.823910 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.823907    2584 scope.go:117] "RemoveContainer" containerID="b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8"
Jul 27 18:16:18.833836 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.833814    2584 scope.go:117] "RemoveContainer" containerID="6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7"
Jul 27 18:16:18.842840 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.842822    2584 scope.go:117] "RemoveContainer" containerID="b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8"
Jul 27 18:16:18.843121 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:16:18.843100    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8\": container with ID starting with b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8 not found: ID does not exist" containerID="b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8"
Jul 27 18:16:18.843201 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.843129    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8"} err="failed to get container status \"b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8\": rpc error: code = NotFound desc = could not find container \"b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8\": container with ID starting with b8524c648295b9aabb75992ed777d837cc69b0887add094785c8b8cc29fe2fe8 not found: ID does not exist"
Jul 27 18:16:18.843201 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.843146    2584 scope.go:117] "RemoveContainer" containerID="6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7"
Jul 27 18:16:18.843399 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:16:18.843379    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7\": container with ID starting with 6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7 not found: ID does not exist" containerID="6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7"
Jul 27 18:16:18.843444 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.843405    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7"} err="failed to get container status \"6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7\": rpc error: code = NotFound desc = could not find container \"6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7\": container with ID starting with 6a8f5241e8fa7e93bbbffccc8cdd5f5244182def2dbc9fae571346127b1f60c7 not found: ID does not exist"
Jul 27 18:16:18.851439 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.851389    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"]
Jul 27 18:16:18.854319 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:18.854292    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-1aa3ff-openshift-default-6575cc44b4-md56s"]
Jul 27 18:16:20.823957 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:20.823925    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" path="/var/lib/kubelet/pods/7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35/volumes"
Jul 27 18:16:23.871962 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:23.871865    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"]
Jul 27 18:16:23.872433 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:23.872115    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs" podUID="a2e369c5-e909-472c-ad18-c6b9646773d9" containerName="maas-api" containerID="cri-o://f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9" gracePeriod=30
Jul 27 18:16:23.910183 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:23.910140    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:24.007817 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.007771    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:24.109061 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.109032    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:24.126944 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.126861    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:16:24.246369 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.246333    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtdxv\" (UniqueName: \"kubernetes.io/projected/a2e369c5-e909-472c-ad18-c6b9646773d9-kube-api-access-xtdxv\") pod \"a2e369c5-e909-472c-ad18-c6b9646773d9\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") "
Jul 27 18:16:24.246546 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.246484    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e369c5-e909-472c-ad18-c6b9646773d9-maas-api-tls\") pod \"a2e369c5-e909-472c-ad18-c6b9646773d9\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") "
Jul 27 18:16:24.246595 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.246544    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2e369c5-e909-472c-ad18-c6b9646773d9-openshift-service-ca\") pod \"a2e369c5-e909-472c-ad18-c6b9646773d9\" (UID: \"a2e369c5-e909-472c-ad18-c6b9646773d9\") "
Jul 27 18:16:24.246977 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.246945    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2e369c5-e909-472c-ad18-c6b9646773d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2e369c5-e909-472c-ad18-c6b9646773d9" (UID: "a2e369c5-e909-472c-ad18-c6b9646773d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:16:24.248730 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.248707    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2e369c5-e909-472c-ad18-c6b9646773d9-kube-api-access-xtdxv" (OuterVolumeSpecName: "kube-api-access-xtdxv") pod "a2e369c5-e909-472c-ad18-c6b9646773d9" (UID: "a2e369c5-e909-472c-ad18-c6b9646773d9"). InnerVolumeSpecName "kube-api-access-xtdxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:16:24.248849 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.248754    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2e369c5-e909-472c-ad18-c6b9646773d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2e369c5-e909-472c-ad18-c6b9646773d9" (UID: "a2e369c5-e909-472c-ad18-c6b9646773d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:16:24.347789 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.347738    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2e369c5-e909-472c-ad18-c6b9646773d9-maas-api-tls\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:24.347789 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.347779    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2e369c5-e909-472c-ad18-c6b9646773d9-openshift-service-ca\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:24.347789 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.347796    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtdxv\" (UniqueName: \"kubernetes.io/projected/a2e369c5-e909-472c-ad18-c6b9646773d9-kube-api-access-xtdxv\") on node \"ip-10-0-133-68.ec2.internal\" DevicePath \"\""
Jul 27 18:16:24.855243 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.855208    2584 generic.go:356] "Generic (PLEG): container finished" podID="a2e369c5-e909-472c-ad18-c6b9646773d9" containerID="f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9" exitCode=0
Jul 27 18:16:24.855411 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.855256    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"
Jul 27 18:16:24.855411 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.855289    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs" event={"ID":"a2e369c5-e909-472c-ad18-c6b9646773d9","Type":"ContainerDied","Data":"f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9"}
Jul 27 18:16:24.855411 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.855345    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs" event={"ID":"a2e369c5-e909-472c-ad18-c6b9646773d9","Type":"ContainerDied","Data":"5e4a53ee9d25c6a4b2abeda0be20ef4a532dfae7aeac78fa88dbd00b3dc23dab"}
Jul 27 18:16:24.855411 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.855372    2584 scope.go:117] "RemoveContainer" containerID="f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9"
Jul 27 18:16:24.864243 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.864225    2584 scope.go:117] "RemoveContainer" containerID="f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9"
Jul 27 18:16:24.864503 ip-10-0-133-68 kubenswrapper[2584]: E0727 18:16:24.864480    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9\": container with ID starting with f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9 not found: ID does not exist" containerID="f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9"
Jul 27 18:16:24.864554 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.864512    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9"} err="failed to get container status \"f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9\": rpc error: code = NotFound desc = could not find container \"f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9\": container with ID starting with f2d898cbe4b99c48c03ab716690ec39af876fa193943c661e14911548fd8ace9 not found: ID does not exist"
Jul 27 18:16:24.872518 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.872489    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"]
Jul 27 18:16:24.876681 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:24.876651    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ed5396-7476779b7-d8rcs"]
Jul 27 18:16:26.823966 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:26.823933    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2e369c5-e909-472c-ad18-c6b9646773d9" path="/var/lib/kubelet/pods/a2e369c5-e909-472c-ad18-c6b9646773d9/volumes"
Jul 27 18:16:28.785186 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:28.785154    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:29.086875 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:29.086829    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:35.560109 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:35.560065    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:35.621523 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:35.621485    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:35.798210 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:35.798172    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:40.143551 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:40.143504    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:40.202181 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:40.202142    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:42.102485 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:42.102457    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b8d4d47-98thz_b6cd0e43-2bf0-4a3a-b409-290e146ff66d/copy-manifests/0.log"
Jul 27 18:16:42.108378 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:42.108354    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b8d4d47-98thz_b6cd0e43-2bf0-4a3a-b409-290e146ff66d/manager/0.log"
Jul 27 18:16:42.548418 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:42.548321    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5d8cd8686-ft867_ed47e6fe-a625-4d4e-9148-c5bfee594724/maas-api/0.log"
Jul 27 18:16:42.574962 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:42.574928    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:42.676449 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:42.676401    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:45.311411 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:45.311376    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx_d2c01468-60be-450a-900e-3d9a6a43ccb0/istio-proxy/0.log"
Jul 27 18:16:45.438624 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:45.438593    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zgrlm_26bbff3a-415e-4487-93a6-9d51305ba00e/discovery/0.log"
Jul 27 18:16:45.901277 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:45.901249    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-kg5kf_0f06b985-67d7-49e9-8a8f-a9e2c62f7367/payload-pre-processing/0.log"
Jul 27 18:16:46.471955 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:46.471925    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lm88q_30ea5c73-1869-42e4-898b-2226f7d9ad5d/main/0.log"
Jul 27 18:16:46.582065 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:46.582040    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-8rd9f_f1777c1b-130d-48a9-8739-8d75b26fd1aa/main/0.log"
Jul 27 18:16:46.915000 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:46.914967    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-vqnpc_5e6acf61-21bd-4b12-b217-7beec8c092ca/main/0.log"
Jul 27 18:16:52.522099 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:52.522046    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:52.592510 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:52.592475    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:56.105264 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:56.105234    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lc6cl_5022fa8c-0fd5-4291-ab27-15b6ddae7ad2/global-pull-secret-syncer/0.log"
Jul 27 18:16:56.214957 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:56.214908    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hvmch_ee67f1bd-8263-43a0-b0fd-d552419d88a6/konnectivity-agent/0.log"
Jul 27 18:16:56.366832 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:16:56.366726    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-68.ec2.internal_47b82ae00a4e61d8c8286ec65e9c9c5f/haproxy/0.log"
Jul 27 18:17:02.940111 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:02.940081    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80ac313-5a55-4e61-be83-2b0cbdcf3e53/alertmanager/0.log"
Jul 27 18:17:02.964570 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:02.964542    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80ac313-5a55-4e61-be83-2b0cbdcf3e53/config-reloader/0.log"
Jul 27 18:17:02.986172 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:02.986150    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80ac313-5a55-4e61-be83-2b0cbdcf3e53/kube-rbac-proxy-web/0.log"
Jul 27 18:17:03.006848 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.006821    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80ac313-5a55-4e61-be83-2b0cbdcf3e53/kube-rbac-proxy/0.log"
Jul 27 18:17:03.030462 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.030437    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80ac313-5a55-4e61-be83-2b0cbdcf3e53/kube-rbac-proxy-metric/0.log"
Jul 27 18:17:03.060345 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.060313    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80ac313-5a55-4e61-be83-2b0cbdcf3e53/prom-label-proxy/0.log"
Jul 27 18:17:03.082568 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.082546    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80ac313-5a55-4e61-be83-2b0cbdcf3e53/init-config-reloader/0.log"
Jul 27 18:17:03.167545 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.167458    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-m4wht_9856cdad-56b2-40a5-b31e-7ad5903ac0a2/kube-state-metrics/0.log"
Jul 27 18:17:03.186593 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.186568    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-m4wht_9856cdad-56b2-40a5-b31e-7ad5903ac0a2/kube-rbac-proxy-main/0.log"
Jul 27 18:17:03.208963 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.208924    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-m4wht_9856cdad-56b2-40a5-b31e-7ad5903ac0a2/kube-rbac-proxy-self/0.log"
Jul 27 18:17:03.237409 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.237367    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-db6bff894-c4tpj_c7ce4b16-6443-481b-b9cd-e96be4186dee/metrics-server/0.log"
Jul 27 18:17:03.267906 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.267876    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-4w8sp_6900e0a3-9406-4bab-8863-542b0fd781ed/monitoring-plugin/0.log"
Jul 27 18:17:03.398326 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.398295    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mchtj_923ea7a6-0b87-46f3-9f76-efcae55c4b60/node-exporter/0.log"
Jul 27 18:17:03.424049 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.423973    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mchtj_923ea7a6-0b87-46f3-9f76-efcae55c4b60/kube-rbac-proxy/0.log"
Jul 27 18:17:03.446765 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.446741    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mchtj_923ea7a6-0b87-46f3-9f76-efcae55c4b60/init-textfile/0.log"
Jul 27 18:17:03.573809 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.573782    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-lg5tp_47880bcc-3ccc-4a21-8942-9a3a95bcbf31/kube-rbac-proxy-main/0.log"
Jul 27 18:17:03.610534 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.610511    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-lg5tp_47880bcc-3ccc-4a21-8942-9a3a95bcbf31/kube-rbac-proxy-self/0.log"
Jul 27 18:17:03.639835 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.639797    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-lg5tp_47880bcc-3ccc-4a21-8942-9a3a95bcbf31/openshift-state-metrics/0.log"
Jul 27 18:17:03.682568 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.682475    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5c1d5554-2844-4eb5-9239-fedc50739257/prometheus/0.log"
Jul 27 18:17:03.705542 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.705514    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5c1d5554-2844-4eb5-9239-fedc50739257/config-reloader/0.log"
Jul 27 18:17:03.732694 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.732638    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5c1d5554-2844-4eb5-9239-fedc50739257/thanos-sidecar/0.log"
Jul 27 18:17:03.756526 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.756500    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5c1d5554-2844-4eb5-9239-fedc50739257/kube-rbac-proxy-web/0.log"
Jul 27 18:17:03.781448 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.781424    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5c1d5554-2844-4eb5-9239-fedc50739257/kube-rbac-proxy/0.log"
Jul 27 18:17:03.804559 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.804537    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5c1d5554-2844-4eb5-9239-fedc50739257/kube-rbac-proxy-thanos/0.log"
Jul 27 18:17:03.826903 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.826868    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5c1d5554-2844-4eb5-9239-fedc50739257/init-config-reloader/0.log"
Jul 27 18:17:03.865767 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.865735    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-jzqgn_24fb7cdb-2ad4-4e28-885c-42887a33e92b/prometheus-operator/0.log"
Jul 27 18:17:03.889800 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.889772    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-jzqgn_24fb7cdb-2ad4-4e28-885c-42887a33e92b/kube-rbac-proxy/0.log"
Jul 27 18:17:03.919194 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:03.919168    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-qh2r2_0968b2c9-caca-4e07-ae42-5422e344e407/prometheus-operator-admission-webhook/0.log"
Jul 27 18:17:04.053821 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:04.053795    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749c48d54-scjkw_f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69/thanos-query/0.log"
Jul 27 18:17:04.079974 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:04.079941    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749c48d54-scjkw_f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69/kube-rbac-proxy-web/0.log"
Jul 27 18:17:04.103587 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:04.103562    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749c48d54-scjkw_f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69/kube-rbac-proxy/0.log"
Jul 27 18:17:04.127083 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:04.127057    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749c48d54-scjkw_f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69/prom-label-proxy/0.log"
Jul 27 18:17:04.155163 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:04.155135    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749c48d54-scjkw_f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69/kube-rbac-proxy-rules/0.log"
Jul 27 18:17:04.178721 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:04.178669    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-749c48d54-scjkw_f9ddb52d-29f0-4d65-83f7-9d8b8e96aa69/kube-rbac-proxy-metrics/0.log"
Jul 27 18:17:06.643734 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.643708    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-67488b6847-sjmwx_8a7802a1-29ba-40da-9764-1308555af6d5/console/0.log"
Jul 27 18:17:06.881024 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.880993    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"]
Jul 27 18:17:06.881395 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881383    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerName="istio-proxy"
Jul 27 18:17:06.881458 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881397    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerName="istio-proxy"
Jul 27 18:17:06.881458 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881406    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerName="istio-proxy"
Jul 27 18:17:06.881458 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881412    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerName="istio-proxy"
Jul 27 18:17:06.881458 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881417    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68786b2a-5f2b-4f17-8ceb-d32053f0def7" containerName="maas-api"
Jul 27 18:17:06.881458 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881423    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="68786b2a-5f2b-4f17-8ceb-d32053f0def7" containerName="maas-api"
Jul 27 18:17:06.881458 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881432    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="963f1f41-6d40-4c97-a3dd-03745d101b6e" containerName="maas-api"
Jul 27 18:17:06.881458 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881437    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="963f1f41-6d40-4c97-a3dd-03745d101b6e" containerName="maas-api"
Jul 27 18:17:06.881458 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881447    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5bdb232-185b-4963-b0a2-099c4549af19" containerName="payload-pre-processing"
Jul 27 18:17:06.881458 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881452    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5bdb232-185b-4963-b0a2-099c4549af19" containerName="payload-pre-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881465    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a88ced35-a900-46e3-b140-759db37bd087" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881471    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a88ced35-a900-46e3-b140-759db37bd087" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881479    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0e4b412-5c96-42a4-948d-1b53f9622f98" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881484    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0e4b412-5c96-42a4-948d-1b53f9622f98" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881490    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2e369c5-e909-472c-ad18-c6b9646773d9" containerName="maas-api"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881497    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2e369c5-e909-472c-ad18-c6b9646773d9" containerName="maas-api"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881503    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f39721d-3a00-4c50-a2db-c666f8e51003" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881508    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f39721d-3a00-4c50-a2db-c666f8e51003" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881515    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerName="istio-proxy"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881521    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerName="istio-proxy"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881530    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbb1321c-2dff-4612-b2b0-d829ab6db438" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881535    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbb1321c-2dff-4612-b2b0-d829ab6db438" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881606    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce1d6f45-055e-4716-9d75-44270fd85b6c" containerName="istio-proxy"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881613    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5bdb232-185b-4963-b0a2-099c4549af19" containerName="payload-pre-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881619    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2e369c5-e909-472c-ad18-c6b9646773d9" containerName="maas-api"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881625    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f39721d-3a00-4c50-a2db-c666f8e51003" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881630    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="963f1f41-6d40-4c97-a3dd-03745d101b6e" containerName="maas-api"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881636    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0e4b412-5c96-42a4-948d-1b53f9622f98" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881642    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="68786b2a-5f2b-4f17-8ceb-d32053f0def7" containerName="maas-api"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881646    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a88ced35-a900-46e3-b140-759db37bd087" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881654    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerName="istio-proxy"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881658    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbb1321c-2dff-4612-b2b0-d829ab6db438" containerName="payload-processing"
Jul 27 18:17:06.881792 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.881663    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8cf84b-8cc0-4d6e-a89d-f9519ffb5c35" containerName="istio-proxy"
Jul 27 18:17:06.886097 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.886077    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:06.889182 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.889162    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dglvr\"/\"kube-root-ca.crt\""
Jul 27 18:17:06.890493 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.890470    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dglvr\"/\"openshift-service-ca.crt\""
Jul 27 18:17:06.890583 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.890473    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dglvr\"/\"default-dockercfg-s4v9h\""
Jul 27 18:17:06.902532 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:06.899943    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"]
Jul 27 18:17:07.078972 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.078937    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-sys\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.078972 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.078975    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-proc\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.079179 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.079021    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-lib-modules\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.079179 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.079067    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-podres\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.079179 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.079113    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr96v\" (UniqueName: \"kubernetes.io/projected/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-kube-api-access-vr96v\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.180592 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.180499    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-lib-modules\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.180592 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.180550    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-podres\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.180592 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.180590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr96v\" (UniqueName: \"kubernetes.io/projected/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-kube-api-access-vr96v\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.180941 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.180653    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-sys\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.180941 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.180668    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-proc\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.180941 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.180776    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-lib-modules\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.180941 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.180792    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-proc\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.180941 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.180792    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-sys\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.180941 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.180846    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-podres\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.189986 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.189960    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr96v\" (UniqueName: \"kubernetes.io/projected/755154f3-ad5b-4adc-8b07-c9cd6ae4eee0-kube-api-access-vr96v\") pod \"perf-node-gather-daemonset-pqxmn\" (UID: \"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.197979 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.197956    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:07.347803 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:07.347771    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"]
Jul 27 18:17:07.349559 ip-10-0-133-68 kubenswrapper[2584]: W0727 18:17:07.349532    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod755154f3_ad5b_4adc_8b07_c9cd6ae4eee0.slice/crio-f4e3a92109c175681353dcc9468994196ad9403de26fd5442cf2c49ceecd55f8 WatchSource:0}: Error finding container f4e3a92109c175681353dcc9468994196ad9403de26fd5442cf2c49ceecd55f8: Status 404 returned error can't find the container with id f4e3a92109c175681353dcc9468994196ad9403de26fd5442cf2c49ceecd55f8
Jul 27 18:17:08.070167 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:08.070129    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn" event={"ID":"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0","Type":"ContainerStarted","Data":"356016d3a90a0086068aebef58670eff21a29011c7dd3e167b3594d1d3519a29"}
Jul 27 18:17:08.070612 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:08.070186    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn" event={"ID":"755154f3-ad5b-4adc-8b07-c9cd6ae4eee0","Type":"ContainerStarted","Data":"f4e3a92109c175681353dcc9468994196ad9403de26fd5442cf2c49ceecd55f8"}
Jul 27 18:17:08.070612 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:08.070226    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:08.090405 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:08.090345    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn" podStartSLOduration=2.090330549 podStartE2EDuration="2.090330549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:17:08.088997603 +0000 UTC m=+3365.797145935" watchObservedRunningTime="2026-07-27 18:17:08.090330549 +0000 UTC m=+3365.798478865"
Jul 27 18:17:08.608406 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:08.608374    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2hlk2_e29ddd7d-78f5-4b66-ad2d-3e5010129212/dns/0.log"
Jul 27 18:17:08.638080 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:08.638051    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2hlk2_e29ddd7d-78f5-4b66-ad2d-3e5010129212/kube-rbac-proxy/0.log"
Jul 27 18:17:08.834351 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:08.834317    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vx8bd_54626d74-62d9-4fea-ae0a-1a73a8f7fee3/dns-node-resolver/0.log"
Jul 27 18:17:09.461526 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:09.461496    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ljv28_3c4df1df-58f8-48cf-b5cc-2ca9669d7f32/node-ca/0.log"
Jul 27 18:17:10.512276 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:10.512247    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx_d2c01468-60be-450a-900e-3d9a6a43ccb0/istio-proxy/0.log"
Jul 27 18:17:10.673024 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:10.672995    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zgrlm_26bbff3a-415e-4487-93a6-9d51305ba00e/discovery/0.log"
Jul 27 18:17:10.858259 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:10.858230    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-kg5kf_0f06b985-67d7-49e9-8a8f-a9e2c62f7367/payload-pre-processing/0.log"
Jul 27 18:17:11.745084 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:11.745051    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4vhhn_d34c6ca9-5a01-465e-bfba-2b9b8d468e94/serve-healthcheck-canary/0.log"
Jul 27 18:17:12.374936 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:12.374902    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nrpkt_3ccfb319-8736-425b-bc66-486e3c249e88/kube-rbac-proxy/0.log"
Jul 27 18:17:12.396766 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:12.396741    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nrpkt_3ccfb319-8736-425b-bc66-486e3c249e88/exporter/0.log"
Jul 27 18:17:12.418969 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:12.418935    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nrpkt_3ccfb319-8736-425b-bc66-486e3c249e88/extractor/0.log"
Jul 27 18:17:14.086169 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:14.086142    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-pqxmn"
Jul 27 18:17:15.566649 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:15.566614    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b8d4d47-98thz_b6cd0e43-2bf0-4a3a-b409-290e146ff66d/manager/0.log"
Jul 27 18:17:15.589040 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:15.589016    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b8d4d47-98thz_b6cd0e43-2bf0-4a3a-b409-290e146ff66d/copy-manifests/0.log"
Jul 27 18:17:16.536425 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:16.536377    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx_d2c01468-60be-450a-900e-3d9a6a43ccb0/istio-proxy/0.log"
Jul 27 18:17:16.687743 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:16.687704    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zgrlm_26bbff3a-415e-4487-93a6-9d51305ba00e/discovery/0.log"
Jul 27 18:17:16.833295 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:16.833261    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-kg5kf_0f06b985-67d7-49e9-8a8f-a9e2c62f7367/payload-pre-processing/0.log"
Jul 27 18:17:17.165986 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:17.165899    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76849bb8b4-l8qt7_9a3c5fbc-a3b4-491a-95c6-fefbd9059dff/manager/0.log"
Jul 27 18:17:17.297545 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:17.297511    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-7gcc6_1c166456-677b-4b37-876b-1c150f50cd98/openshift-lws-operator/0.log"
Jul 27 18:17:18.326263 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:18.326233    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6wkpx_d2c01468-60be-450a-900e-3d9a6a43ccb0/istio-proxy/0.log"
Jul 27 18:17:18.485927 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:18.485835    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zgrlm_26bbff3a-415e-4487-93a6-9d51305ba00e/discovery/0.log"
Jul 27 18:17:18.622741 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:18.622712    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-kg5kf_0f06b985-67d7-49e9-8a8f-a9e2c62f7367/payload-pre-processing/0.log"
Jul 27 18:17:23.769233 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:23.769199    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kbt4b_f02fbba4-deae-4855-8e20-88163660aa6e/kube-multus-additional-cni-plugins/0.log"
Jul 27 18:17:23.791407 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:23.791377    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kbt4b_f02fbba4-deae-4855-8e20-88163660aa6e/egress-router-binary-copy/0.log"
Jul 27 18:17:23.814995 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:23.814960    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kbt4b_f02fbba4-deae-4855-8e20-88163660aa6e/cni-plugins/0.log"
Jul 27 18:17:23.838562 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:23.838536    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kbt4b_f02fbba4-deae-4855-8e20-88163660aa6e/bond-cni-plugin/0.log"
Jul 27 18:17:23.862043 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:23.862009    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kbt4b_f02fbba4-deae-4855-8e20-88163660aa6e/routeoverride-cni/0.log"
Jul 27 18:17:23.888112 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:23.888083    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kbt4b_f02fbba4-deae-4855-8e20-88163660aa6e/whereabouts-cni-bincopy/0.log"
Jul 27 18:17:23.918091 ip-10-0-133-68 kubenswrapper[2584]: I0727 18:17:23.918058    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kbt4b_f02fbba4-deae-4855-8e20-88163660aa6e/whereabouts-cni/0.log"